I got it working! Two things:
1. I did have to contact steam support to get matchmaking enabled on my app id. My app was not created as free, but matchmaking was still disabled. Maybe this is the new norm for steam, as in 2017 when I created my last app I didn't have to take that step.
2. I had the SDK installed in D:\Steamworks\sdk but in engine I simply had the path D:\Steamworks set as the SDK root. If anyone else has this problem make sure you're pointed at the folder that has the contents of the SDK in it, not a folder that has a folder that has the contents :P
Thanks again for your help Yellow, and thanks for the extension - you rule!