Zip lines is something that is on my list for a long time :D As for ladders and climbing, if everything goes ok, they work, but if someone tries to jump off them that's when things may break (it's easy to desynchronise vr space from game world's space). I still need a good solution for that. I have a few ideas that I need to try out.
With ladders you're standing in the same location whether you are holding or letting go. Perhaps the problem would be if someone is holding the ladder then walks away. Most games the play space will follow you in that situation but in your case they should lose there grip and fall landing within the correct coordinates.