That was the weird part, I was in a relationship with Lilly in season two but kept losing her when proposing the harem with Celine and thought I had to make it up later in the story.
But I finally found it. When Lilly finds out about Celine I always continued with both of them. But I had to then tell her about the others. My mistake.