Ok, so I (maybe) fixed it, at least for now. Not sure what did it, but in the end I reinstalled my drivers, updated windows, rebooted properly (not a hard "press the power button" reboot like I had to when the PC was sluggish), and also passed the game to full screen before doing all that (which with a slow game took a good 3 minutes).
Now it works, at least for the one session I did so far since, but I can't be sure that'll stick as all my first play sessions were fine too before these steps, so maybe sluggishness will build up again over time ?
Anyway, hope it helps either dev, players, or both.
Thanks for this awesome game.