The physic damage happening isn't really my problem, I had saved shortly before going up, I'm just confused on why could my character just start falling when I didn't do anything to trigger it
There is a hunger side effect that causes you to ragdoll. Probably just got unlucky it happened as you climbed the ladder.