Very interesting concept. It feels like walking in a non-euclidean space, it's a maze shaped in the form of dividing paths.
I know I could use pen & paper to solve the puzzle, but I think it'd be nice if there was a way to do that in-game in some capacity. It would then start as an exploration game, and then become a "calculate the path" puzzle style game. Maybe after getting the 5 gems, the tower would start to collapse and you'd have to get out of it quick. At that point, knowing where to go could be trivial, and the challenge would be executing the path without getting confused about directions. But I digress.
Unique interpretation of the theme, I like it. Good job!