I have experienced a problem a few times of, say, I leave a slave in the stocks as a punishment, but someone lets them out and they just start wandering the prison and not returning to my cell.
I don't see this as an inherently bad thing. In fact, I think a feature where they have a chance if doing that if they have low obedience, and you have to go out, find them, and force them back to the cell would be interesting.