I've been having the same issue with a friend. I know there was already some light shed on this bug but have there been any updates that may have fixed it?
It's definitely somehow caused by the player walking over the pit tiles, but I have no idea why. The main problem with the bug is I can't do any trial and error to fix it cause I personally don't get the desync with anyone I've tested with for some reason, though it definitely happens for certain people