It's kinda linked to the story. She's worried at some point and you talk to her in room to help her relax, but there is an option to go to the hot spring instead which has a warning of locking you out of the repeatable scene if you choose it.
Sounds like I probably locked myself out then; will have to see if I can find an older save or edit the save file.
