Yeah that's exactly the problem. I tried hiding some of the effects in the customization view but I guess since they're not on camera and visible Godot doesn't bother compiling them.
Viewing post in Super Serious Battle Tops jam comments
Yes, they must be in view. I had one material off screen and when rotating the camera in it's direction, it lagged the first time. I think that with dynamic lights, there are multiple shaders for the same material. I am not sure about this, because it was couple of months when I first looked at precompiling shaders on Godot.