You either need to get his affection to 45 or 0. Afterwards, you can talk to him and there should be an option to a good ending if you haven't figured it out yet.
