It's not a bug, moving around is the way it works right now.
All I am saying is that if the character could move up or down without going left or right, it would feel a lot more natural and intuitive.