You should make a way to release slaves, so you can swap them out easily. I screwed up and put 3 thieves in the brothel, when I finally captured a mage it was full. Is the current limit 5 in brothel and 5 in houses (if you donate a slave to Saffron instead of fathering a child yourself)?
Viewing post in Black Garden (Alpha 0.1.9b) comments
It's something I'm considering, but don't really think it will happen any time soon. The Brothel should be able to house 6, or 5 if you got Furo in there and didn't save him. You can release slaves which you have in your inventory though. Also, slaves in the brothel are mostly an aesthetic thing, they don't have any other functions aside from getting Goblins pregnant. Only House slaves are able to teach skills, ect.