You're talking about Esto Yensa, where you're supposed to find a Rodenian Leystone after having obtained the Astral Key, right? I think I encountered the same bug when fooling around. If I'm right, the bug only occurs when you visit Esto Yensa too soon, that means after obtaining the Astral Key, but before triggering the dialogue (was it Mizuda in Marshtown?) that steers you towards Esto Yensa and Osteinchyl to obtain the Rodenian Leystones. If you visit Esto Yensa too soon, you can simply walk in (without a cutscene taking place), and then the bug occurs when trying to switch parties. If you do things in order though, a cutscene will be shown when you enter Esto Yensa, and you will actually be prompted to select two different parties before proceeding. Hopefully, you didn't encounter the bug despite doing everything in order and selecting two parties, because then I don't know how to avoid this bug. Also, I obtained Esto Yensa's Rodenian Leystone first (before the one in Osteinchyl), which might or might not make a difference, but at least I was able to switch parties inside Esto Yensa and to complete the dungeon without any further issues.