If anyone stumbles upon this, for the web builds you need to set the Playback Type to "Stream" for the audio to work. Neither "Sample" nor "Default" work for me. The same goes for Polyphonic AudioStream, you need to pass the "AudioServer.PLAYBACK_TYPE_STREAM" in play_stream for it to work properly, just setting it in the editor wont work. I have tested this on Godot 4.6.3 stable.