Is the Bandit's Camp morale above 25? Do you have 40 likes with Harold? Did Barrel already visited you when you were about to sleep in your new home in Bareshade?
You need to show an Emerine stone and a Glafire stone to him. You get the Emerine ones from their hideout at the cave complex, left door, need high perception. Also I think you get the Glafire stone in Mythshore or the cave prior?