Yea, turns out the shader I've used for recoloring and some extra flare was a REALLY expensive shader to run... I've now fixed it, but due to the game jam's voting period I'm unable to release the fix.
If you'd be inclined to play the game with better performance I'd recommend checking it out after the fix is released.
Sorry for overlooking that.