so basically what you're saying is that the game is unplayable with a controller. I am having the same experience, unfortunately.
Well not exactly just use the joystick instead of the D-Pad. I have also found out you can hold a direction after fleeing/winning and you should be able to move one grass tile at a time.