Yeah the timer is a little too long, you really need to remember where is everything before it change back. But since it is tied to the music I can't really change it now :)
I have made the Hitboxes really smalls so it doesn't feel unfair but it still quickly get difficult.