I think that's plenty clear, but I also believe it would be better to integrate that into the choice rather than put a stage direction there. "He only shoots you a quick smile, you aren't as close as you used to be." no other backstory explanation necessary in that moment as time is of the essence and everyone's mind is not really on childhood memories and nostalgic recollections.