I think I found a bug or something (I don't suppose this should happen), and I managed to remove the permanent chastity cage using estroGplus (feminizing pill), as I say I don't know if this was intentional or just something that was overlooked when that content was added, I leave a screenshot as proof
11 days ago(+2)
Well, I just remembered/re-found (haven't played since 0.1.6) that this might be intentional or something like that, since unless rahi has confirmed it as a bug then this was intentional 
(this is from the developer comments on that same quest line)