So, I got to meet him, really nice fella, I haven't tried another way but here's what I did if you want to meet him, you need to fully earn Kirby's trust, deal with Mrs Kimura's offer, what I did was talk to Joseph after hearing her offer and asking for more time to think, he'll tell you to go for a 50/50 deal (If you accept the deal without the 50/50 Kirby will leave the farm), that way everyone's happy and then you can ask Kirby about Robert and she'll take you to him.

Don't know if it works while declining the offer, haven't played that route.