you're stuck doing a quest that has equipment. If you do annoying mission, you have to finish it to remove it. Alternatively, you can just talk to the quest giver and they can cancel the quest.
No, other stuff gets (visually) stuck on even after it's "removed" through the inventory. For example, I found myself with a dog mask and dog ears stuck on. I could equip a different mask and then I'd see both of them at the same time, and if I actually equipped the dog ears I'd see that I'm actually wearing two sets!