It is but it might be if you've submitted/seduced him too much he appears more? In his line it says he can smell you from miles away so maybe the more you like/increase his affection the more he appears?
Hmm that might be the case, since a lot of times he kind of restricted my actions a lot whenever the encounter happens.
Alright we possibly figured it out!
