You have to click the cursed gem again to disable. I know it's pretty unintuitive. I lost 14 hp to it before I figured it out.
