This is strange, I'm not really sure. I can F11, but the itch.io full screen button on the bottom right of the screen still stays visible. I imagine that there's something funky going on with how input gets handled between my game's canvas element and itch's container.
For now I suggest just pressing the bottom right expand button that itch has provided!