If you notice, the husband often masturbates multiple times imagining his wife being fucked by other men. He also masturbates a lot knowing his wife is having sex with Yuma, imagining the positions and the moans.
I feel like the husband is more aroused by watching his wife have sex with Yuma than by having sex himself with Hana. But I also agree that he should have more scenes with his wife and shouldn't be such a spectator.