I've been having the same issue for the last several builds. Nothing I've tried seems to prevent it, the game seems to get confused because an invader doesn't despawn so it thinks you're still in combat.
