Tbh Id say to keep the ladder part, it DEFINITELY is rewarding when u actually get the rhythm.
It already have a respawn close to it and if u dont die you are exactly at the start most of the time, so you can just try again!
Just some more jumps that prepares the player for that would be better!