Everyone's feelings are going to be different on this, so for a sort of nebulous answer:
If you'd sell it as "a visual novel with puzzle elements" (999), versus a "puzzle game with visual novel elements" (Professor Layton), then it's a visual novel.
As a personal opinion, I think at least 50% of the gameplay should be visual novel component for it to count in this specific instance.