To an extent, the idea was that part of the puzzle is figuring out how to bring the notes places without accidentally putting them in the wrong spot, or without blocking yourself from continuing (seen as early as level 2). But if that's too annoying, we should probably rethink that.