You don't have to wait for your past self to search the solution. When you know what to do you can just redo the actions with your new one, create a new past of yourself a lot quicker.
But it's good feedback, a short tutorial that gives players time to understand the ghost and what they can do with it would help avoid a lot of frustration. (you are not the first with that comment !)