I think there may be more going on here. I also have 8gb VRAM and a 12 core processor and my fps is significantly higher on performance mode. 100+ comfortably.
It was a 24 hour challenge so optimisation work was limited, but it's playable; comfortably so for a spec like you are describing. As I say, there may be something more to this such as driver issues, or your pc may have got caught up compiling shaders and you need to restart the game a couple of times (these are just theories).