I had what felt like a huge speedup when night fell and the vision range dropped.
Also when driving far enough away from the start location to unload the chunk.
I also got the impression that resource gathering was causing the slowdown, so between the two I'm wondering if it is proportional to the number of auto-collecting resource instances you've seen.