I mean, scientifically speaking, you can actually just "get over" phobias and it's a thing people regularly do. Even so, the question is "should someone have to conquer a primal fear response in order to play the game?" and "no" would be a pretty reasonable opinion.
Being that the game is in development, the caves aren't even remotely important to progress, and basically the only way to know about them is to look it up on the wiki, it's probably best to put it on the "nice to have" list and get around to it later on.