It matters which cards you flip, and in what order. Some characters affect face-down character cards, some restrict what you can flip, and you also score more points the fewer cards you flip. Flipping all of them at once gives you a big disadvantage.
A hidden shortcut (holding shift while clicking, perhaps) would be a good idea for earlier levels, when it isn't as important.