A couple of the side quests are bugged to not clear off, but for that particular one:
- Go to Leah's Mansion
- Use 3 souls to repair the Garden by handing them to Sophie. (Left room from the Entrance)
- Have the Dryad lady move in.
- Talk to her until the dialogue repeats (she'll explain how to plant seeds and grow plants, etc)
- Plant the fruit seed in the tiny square of land in the garden.
- Side quest should clear.