Maybe your thumbstick is drifting. The default "Increment Origin Forward" (zoom) keybinding is the thumbstick up/down. You can try unbinding that keybinding to see if that stops the issue. If that's the case, you can adjust the deadzone in your Quest 2 settings under devices, then controllers.