Quite an interesting idea. Do agree with a lot of the other comments that it feels like it's a game where you need to learn the route as you go and 'speedrun' the game - nothing wrong with that, just a different style of game, and something the player needs to keep in mind.
I did run into an odd ... bug? Double-jumping against a wall makes you get 'stuck' to the wall, stopping the double jump. So to double jump up that starting area, for example, you need to stand away from the wall; if you double-jump against the wall/while running into the wall you'll never get up. Took a bit to figure that out xD