I assume you are using raycasts for the collision detection I think you can input physics layer filters to the raycast function and then you just need to add a new physics layer to the planet so you can filter it out and still have the player's and asteroids collide with it, just not the raycasts.