Gigi is not in the candy factory! She can be found again in the Dragon Temple, west of the town in Floater's Forest. The Candy Factory can be entered after defeating the Seal of Flesh boss.