You need to have the bandits low morale so they start attacking you in places again then when you rest at night there's a chance for them to attack and he fights them off.
