For me the lag only kicks in when I have returned to the menu a lot. I think when you're changing scenes the other scenes are all still running. That's a common mistake in godot.
oh thanks! I will be checking that, Yeah probably that's the reason. :]