This is a really cool idea, but I feel like it could be refined a bit more.
Building a mental map of the game was a very interesting process. The shifting rooms caught me off guard, and the game felt much larger and more maze like than I expected as a result.
My main issue is that there wasn't really much incentive to risk exploring rooms since often going right back through the door you just came in was the correct path. I'm not entirely sure how you could fix that (or if it's even something that even needs fixing), but maybe you could have an in-game map system that lets players try to piece the world together themselves, or a powerup that gives the player the option to use doors 'normally'.
Overall, awesome idea, I'd be interested in seeing how far you can take it.