Thanks for playing, unfortunately because I left everything so last minute I never got around to being able to destroy buildings, in the updated version I've made for after the GameJam submissions are over I've already implemented it, but I really should create an in-game help or something similar as nothing is really explained in game and quite a few people are having trouble with it, as it's not self explanatory.
Thanks again for the feedback!