Oh definitely not! I don't recommend playing with a touchpad.
The balls do get some invincibility frames so to say, right after you click them. Maybe that's what you experienced?
Well, the heart-shaped cursor does make it kinda difficult to see where your click is, so it ends up missing more often than not (even when it feels like it should be connecting...)
Out of the 3 minigames, I struggled with that one way more than the other two myself, since it took a solid 30 minutes of attempts to get it TwT
Maybe making the ball hitboxes a bit larger would help?