Have you kept up playing video games with Adrian on Tuesdays after paying for Adrian's trip? You should eventually get dialogue with him talking about how Anna & Josef like you more, answer nicely and he'll say you can visit him anytime.
I don't remember if this happened before I went the Alice Sharing route, but it definitely doesn't require Anna sharing.