Check the colliders on those walls and maybe turn down the mass on your character so they don't fall so quickly.
