Make sure all quests are done and all girls have no level up event. Then try to go to sleep multiple times.
