Step 1: Go into Edit Mode by pressing "E".
Step 2: While in Edit Mode press "delete" to go into Delete Mode. ("delete" = keyboard button with delete on it)
Step 3: Click on the building you want to delete.
Don't forget that you have to go out of Delete Mode first (by pressing "delete") before you can get out of Edit Mode (by pressing "E") or you can just click on the "X".