Bandit's camp morale is low (You can see it when you are on the camp dot on the map.) If the morale is lower than... 20 or 25 then Barrel is not appearing in the town anymore and bandits are attacking on the track again. You need to do some quests or bribe them to increase the morale. The morale getting lower every day.