If you press escape, you get volume sliders were you can turn down music, or mute it altogether. I know this is not super obvious, but I couldn't do any better during the jam period. Hopefully the post-jam update in the weekend makes things clearer. Thanks for playing.
UPDATE FOR FUTURE READERS: There's also a button at the bottom-left of the screen to access the settings page.