I reckon a few people had trouble with this game because it's not well telegraphed that you can travel north, east, south, and west.
The game itself is something like a puzzle, and thus, by nature a bit cryptic. In that case, all the more care should be taken to make everything clearly communicated to the player. Smooth controls and cool concept!
Keep up the good work!