Thanks for the feedback! By now I've realised this movement is something I've just gotten used to after hours of testing, but is indeed pretty bad. Definitely going to try to account for that next time I attempt something like this.
I hope the game still feels at least somewhat playable despite that though!