the way we did it was adjusting camera FOV, lowering it when interacting, and then bringing it back up when leaving.
