I think I figured it out. The first time we stay to help him at the bar, we need to choose a "strong drink" and then ask him "about sex," otherwise further interaction is blocked. Just like with Prince Keyris, we need to answer "dog" when he asks who we are to him to open the door to romance.. It was easy on the first playthrough, I didn't even think such choices would have such a strong impact. On the second playthrough, I want to pick up Keyris, but it's humiliating