You can engage rotation-only dragging by holding R2 (gamepad) or CTRL (keyboard) while moving fossils. This should fix your issue! Cheers.
