Yes sadly that's a problem with the Godot engine :( I will see what I can do to improve it.
I use Godot for 3D as well and I remember having issues with performance in one game because of the import settings with textures. Maybe this could be the reason for this game too.
Its not because of that, its because of occlusion culling is not supported in Godot
