Once you unlock the underground city (and get the dragon core) + progress Robin enough you'll be able to get rid of it.
Where to get the dragon core I'm struggling On that part Currently
You find it in the underground basement. Its in a chest though, so theres no glow