Now I know this was an LD, so this isn't a criticism but...
If I were to guess, I'd say that somehow you're miscalculating mouse location? The UI is barely functional (At least for me) and where I click sometimes works perfectly, and sometimes has only a passing relation to what happens. Anyhow, it's only a guess. Good luck cleaning this up... if you decide to do that. Or if not, then good luck with whatever other endeavors you choose. :)