is there a way to get affinity for shops we already are 'sold out' in?
You can always battle the seller in duel to increase affinity :)
Except that won't work with Rafaelo or the Bandit since you can't duel them.
You can find them somewhere in Helgem, and also can use the dictionary once you've found their weaknesses.
I think the dueling to raise affinity is unbalanced I dueled Rafaelo 6 times I hardly made any progress. At this point I need to duel like 50+ times for several characters
