Both Maggie and the journal mention your home cave. Make sure you finish the minotaur questline by talking with chief Stoneaxe after beating Mudbrick in wrestling.
I am having this same issue. I have done the wrestling and now the chief says he is wait for me to finish preparations and I have Elly, Mona, and Naka's affection bar past both hearts and up to the lock, and am using v 0.23, but when I go back to the cave I am just told I need to make friends. I have also done all the side quests the girls asked me to do. Not sure what else to do.
Not sure if there was a way to put it in the comment, but here is a drive link. https://drive.google.com/file/d/14qs35rPXXiWxb5UM4BWccq25ldR6j8lR/view?usp=shari...
Thanks for the saves! Yeah, there was some weird behavior with that quest and I made a fix. Please download two files here and put them into your game's data folder along with other data files. Confirm the overwrite, then load your save and check your home cave again. The fix works even if you are in the cave from the start, so you don't need to prepare any other saves or start the game from the start.
Please ping me back if this solution fully solves the problem and I will roll it out to the public update.