Alt + Enter ingame, OR
Right click on the .EXE -> Create Shortcut -> Right click the shortcut -> Select Properties -> Select Target -> ad this: -windowed -> dont forget the space at the beginning -> Run shortcut and profit lol
(On Windows 11: Create Shortcut is located at "Show more options" when you right click the exe)