I guess I made it a little bit hard to get. Doing a "Lost Friend" quest gives the most points. You can get some when you don't take payment after the incident with sickness. Except of that you just need to talk with him, there's like 6 different topics and some of them can give you points. If you have enough then he will stop you when leaving the shop.