I completely agree that the current state of the level swapping is too slow. I plan on giving the game a "post-game jam" update at some point after the voting period ends that would fix the timing of stuff like that and maybe some other features.
Thanks for checking it out though!