Thank you so so much for the feedback! It is really useful and greatly appreciated!
Your suspicion was right; it's all a single scene. I have no idea how to optimize a game and I'm most definitely a begginer when it comes to technical knowledge; my focus usually ends up circling around game design especially, followed by art, music and sound. The situation is so bad that I had a massive frame-rate drop at a very specific point which I only managed to half-fix 20 minutes before submission.
So one question; by external layouts you mean something like tilemaps, right? (I've been wanting to get into learning how to do them properly...)