Yes, it’s true; the replayability is annoying because the progress is rather slow, and as you say, having to take so many steps to get from one place to another makes it less enjoyable, especially if you replay it.
Personally, I have an injector that allows me to modify the game’s clock and make it run three or as many times faster as I want; so I’ve found the replayability entertaining and decent. But if I wanted to do that without the speed hack, I’m sure I’d be constantly clicking on the screen, even with fast-forward dialogue enabled.
I really like the game, but I definitely feel that the replayability is tedious. I agree with you, and I hope you take the feedback into account, without any hate or anything like that.