The game by default should hover over the last save file that you saved to... if it is not doing that then I got no idea why because it always does so for me.
The quest reward for bringing in the thief is only 10 Silver and 10 Morality points, plus it is a repeatable quest! Also, yes, any prisoner remains with you until you give them to someone or release them yourself.