If you have finished the Bandit's Attack story, you need to have the morale of the Bandit's Camp to at 25 at least. Then in the evening at the Tavern in Bareshade you can encounter Barrel there.
Adding to wildbara's comment you can also use the love potion on Harold so you don't need to pay for the sex and that doesn't change his relationship with you
