I didn't notice any jitters or popping in my testing, maybe there would be settings to fix that.
Did only the outlines touch? Those are actually outside the collider, only the actual planet accurately represents the collider. I should have changed that. I added the outlines later and didn't even think about the colliders not being accurate anymore.