Actually looks like this can be fixed by disabling the "optimizations for windowed games" as was recommended in the other post: https://itch.io/t/4513801/large-frame-jitter-while-having-a-very-small-extra-win...
Would be interested if this can work out of the box, but disabling that setting is sufficient to unblock me for development.