Is there a way for the npc to not follow you? I keep getting the "following you" text when I never signaled for a follow.
There is no option for it, but there are a few tricks.
One way is to walk out of Silverpine and they will stop following.
The other way is simpler. You can talk to them again right after they follow and force you to leave the conversation. It'll be the same conversation you left, so you can continue it.
If you want the NPC to stop following you and you don't want to talk to them, just talk to them and leave the conversation. (they won't mind unless they are trying to talk to you intentionally) They'll always stop following after you start talking to them again.