Ah got it, you're right, ESC does work. Though if you're playing in full screen, it will first exit full screen, and then you have to press it again (that's why I missed it). But a button would be good :
