So after each assult all characters are free to leave?
They might develop feelings for you for some reason, or perhaps they simply become unwilling to leave you.