After examining my map for a while, it looks like it's possible to get stuck if you collect the bottom-left crystal while upside-down. Is that what happened to you?
It's not a method I had considered for getting there, but at least it's easy enough for me to add a way out without breaking anything else.