Wacky idea! I have to admit that when it started I was skeptical. Then when the second and third numbers came in I went "oooooh. Slick!" I do think you should try allowing the number keys rather than mouse: it would feel a lot more "arcade-y". [Oh, yeah, I see others have suggested the same].
Anyway, I spent ages playing this, really clever idea.