Thank you for the feedback, Baum!
I will look into making the character stop immediately when letting go of the joystick or keyboard key.