Took me a second to get it but once I did it was super fun. I think there could be lots of different puzzle-game applications for this kind of movement. Super cool and very impressive that you did this in a few days!
Edit: Just saw a comment that suggested a compass and I totally agree, I think the only criticism I really had is that it was little frustrating to search all over the map for what you needed, especially if you made a mistake and accidentally slipped off and reset everything. Still really good though!