I haven't made any visual novels, but I'm a software developer, and the one thing holding me back from making one is the art. So, in my opinion, the hardest part is the art.
probably the art, while coding the visual novel can be a little boring and repetitive, finding the artist to help can be even worse, and on the small chance you do find a good one they cost an arm and a leg lol
Holy moly, well it is deserved, some of this visual novels looks so amazing, I think coding the visual novel must be the shortest part of doing a visual novel, meanwhile the art could take a lot more