Hey there!
It turns out that option was accidentally removed during a code merge. If you add +windowed 1 in the command line options of the launcher it'll give you windowed mode!
Naturally, I found the answer half a minute after I made the comment.
However it would be nice if a -jukebox option was available that does the same. (if it's possible to do)
A tad annoying that there are no taskbar buttons in windowed mode to minimize it. But I guess that's just how they implemented windowed mode, way back then.