Stopping to play the song: Yeah, I realized that when it was too late for me to change it.
Z to jump: I'm pretty sure most other computer platformers I've played use that as well. (E.g., Freedom Planet has Z=jump, X=attack, C=special by default.) If your browser supports it, it goes by location on your keyboard, so the key to the right of the left shift key will be jump, even on a German, Dvorak, etc.
Edit to add: Oh, and Z is intended to be used with the arrow keys. If you're using WASD, J might work better (control scheme from the Distorted Travesty series).