I think I found out why you can't get into the Goddess Temple. If you have all three sacred items and put them together, it activates her boss mode. So you have to meet the Goddess first before combining the items.
