Husband Points are gained by acting nice/husband like towards the ladies you married. For example inviting them to a play or celebrating the lantern festival with them. Marital duty however just takes a lot of time to recharge (haven't checked it precisly but maybe a week? Two?).