Web build worked for me after clicking SharedArrayBuffer support on itch edit game page. I got a bit stumped on it too, but googled and fixed it in time. This video was helpful for me:
Dude you are my hero I never had that issue before.
Yeah it was my first time exporting to web on Godot 4 too.