Right now the only thing that counts for your final score is hits taken and time, destroying buildings doesn't contribute to it.
