The ledge grab's hitbox probably needs some tweaking and a better way to circumvent for sure! (if you press Down you won't grab the ledges and you'll fall faster, controlling the character gets a bit harder when doing this though)
Ledge grabs are really finnicky to tweak but i believe that once it's configured right it becomes a ton of fun to use ^v^ I'll just have to experiment more on those types of things to really limit the "control being taken away from the player" thing you said !
Thanks for playing ^^