Keep spending time with him during the day after VA classes. You're missing cut scenes from him. At this point, it matters more what day you spend with him than the amount of days that you do. Think back on what day you don't spend with him and spend time with him on that day the following week (i.e. if last Wednesday you took extra VA classes/did a job to get money, make sure to spend next Wednesday with him). You won't be able to be lovers with him unless those stars fill up.