Yes, it's the shadow route, and don't worry! I was also a little slow, and more when you have so many endings.
To get the ginger star, you must do the entire polyamorous route (keeping the vanilla brooch) and first talk to Ginger to go to the library, when you find Vanilla deciding to leave him alone. Then you return with Ginger and he offers you to go to the city, you talk to Shadow and they go to the place. There you complete the minigames and when you meet Vanilla you stay out of the store, you won't talk to him. And so you unlock the star.
If you don't know how to do the polyamorous route here I explain ;3
First you must inagar more, and when you get to the shadow garden go back. After talking to vanilla, consoling him, and agreeing to go to the library, you must grab the book. The next day make the proposal and answer shadow's questions correctly, in the closet, stay but don't take off the brooch. What follows is not important. As far as they ask you to choose a drink. You must touch a kitchen shelf, one that is above, by doing this it gives you the option to give them like an alcohol.
Hope that it works for you! Sorry if I had any mistake.