Yeah, because terrain collisions are done GPU, but physics runs on CPU. The actual math is trivial.
Okay, final thought. There's a lot of talk about different gameplay elements, enemies, etc. and I had an idea: what if the terrain itself was a sort of "enemy"? Like, it's sentient and tries to harm you. Maybe not all the time and not in all areas, but I think it'd be an interesting concept. I don't think any games have really done something like that and this is the perfect setup for it.