Ok, because everyone is (rightfully) confused, the door to the second floor area is in the back of the temple almost as far as you can go.
You have to go DOWN through a gate and it looks like you're gonna walk through a wall, but you just go through the door.
I swear I ran around the floor for like an hour till I noticed.