There's one even when you can encourage or discourage Logan to go to the bandit's camp. It's after the bandit attack the Bareshade.
If the thug is in the tavern then you had to have the decision with Logan. It's needed to move the story with bandits to go further.
Try to talk to lumberjack, there should be some conversations about what happened. Also if you see a scene between lumberjack and thug in the sawmill during the night then you encourage lumberjack to go to the camp.