This is a good idea, and something I've thought about before. If I end up making a sequel, I will almost certainly add 'temporary powers' like this. Maybe have specific levels designed around a temporary powerup.
It's a struggle I've had with a game like Just Jump. I love powerups and earning fun new abilities. But Just Jump is very calculated in its gameplay. Earning a double jump ability or a dash ability sounds really fun, and I love the gameplay it introduces. But once you unlock those, you would have a huge advantage in previous levels, and it would warp how the leaderboards work. But by having these powerups isolated to a specific level (or as you mentioned to a specific block so it could be limited to certain parts of a level), you solve that problem.
Honestly, I could even mess around with making 'DLC' levels for this game that could mess around with temporary powerups/abilities.
Thanks for the comment! Glad you're having fun playing the game.