I absolutely love the idea here!
The level of polish is also quite good for a game jam (both the story-intro and the mini-map showing the hero are well done).
The difficulty ramps up fast though, and it all very much seems to be designed around a specific set of inputs that the game expects the player to have memorized. (Not really my cup of tea … though a lot of people seem to like that sort of thing, so maybe you’re on to something.)