I can't be sure about this, but it seems to me to happen more readily if you have your turn ons set, and the NPC matches those.
