go to him at 20:00 and he should invite you in for the night. this requires a certain amount of hearts though so can you tell me how much like and what amount of hearts you have with him?
alright, going to his house at night isn't possible at the moment then, have you done the date with him at the bar? also has he invited you to visit the hut with him on Fridays? if you don't have the hut you can unlock it by doing his quest about searching for his friend and looking for it in the forest.