I had the same problem under VICE. Peter jumped when I pulled the joystick down. More specifically, when I released the joystick. Joy up did nothing.
I had the Mouse (1351) installed on port #1. After switching it to joystick as well, the issue went away. So the mouse causes issues in the emulator as well.