ah, weird! thanks for letting me know. could you tell me if pressing X/Escape to bring up the menu works?
I can confirm that pressing escape brings up the menu with the save/close options does work. For me, the arrow key issue is that the character is always moving left. I can momentarily stop them by holding down the right arrow, but when I let go they start going left again. I can use the up/down arrows as intended. Let me know if I can provide any other info! I'm running the game on a Windows 10 machine if that helps the debug process.
*Update*
Plugging in an Xbox 360 controller has fixed movement for me. I know that's not available for everybody but more data for fixes!