The controls are a bit borked. If you go left or right, and THEN go up or down to go diagonally - you wil continue to go diagonally even if you stop pressing up/down.
