Maximum affection with bandits don't matter as far as I know. My character's a total slut (to everyone including bandits) but still had 2 hearts with Logan. I guess you're only option now is to invite him to the Fair (with all the requirements done) to gain his other heart. Though I don't know if it'll work since I haven't chosen Bernard in all of my saves during the Fullmoon event so... results may vary.