You have to raise camp morale in the bandit camp, then go in the tavern to trigger the fight in front of the tavern