The vendor option for Custom NPCs isn't ready yet and still being worked on. (Hover mouse over the option while in Routines) It's not supposed to work at the moment.
They don't sell or buy anything in their Trade window. But if they have a Trade window selectable, you can use roleplaying commands to barter with them. They are much more susceptible to trade-related commands of that kind when they are in this state.
