Hm, not sure if this is what's happening with you, but I think people instinctively assume that one click is one pendulum swing. To get your pendulum up to speed, you should kind of spam click, and then find the right rhythm to make sure the pendulum's turning point is right on top of the two marked spots.