Another thing you can do... I noticed if I was 'dirty' (not fresh washed) that it would be harder to convince them to do anything including affection. You can also try to get your charisma up first that will boost your chances.
3 Books boost Charisma also that you can read.
Hope that helps.