I don't understand. When I lose it always says "Enemies Potted: 0" while I pushed some of them in the holes. I understood that I must avoid to destroy the houses but it looks like it's actually just a joke of the lore, and anyway it's too hard not to crush them accidentally. I have no idea what is the gray icon under your coins, a compass?
But the idea is funny. :)
Also the speed difference of your ball for different bar level feels a bit too sharp, unless maybe it's just the max speed that is so fast it feels sharp, but that's just a feeling. But I think there is some room to make the 'speed' more juicy. Or maybe I was just playing too small to avoid the houses so it felt too fast...