Ah, I think I know what you're talking about with the controls. I had a similar feeling and spent some time making the direction derived from where the camera is pointing and also thought that felt weirder so I switched it back. I haven't made a camera system like this before, so I might need to look at more examples.
And sorry you got stuck :( There are a couple bugs like that I wish I had more time to address.
Thank you for playing though! Glad you enjoyed it!