Affection is a hidden stat that is based on your choices.
You can generally gauge it based on whether you are getting the his pov sections or the occasional more romantic variations of scenes.
