Agree. it looks Cheaty and Un-Natural.
But, No, Don't add "Construction Time" on each building
Because, it cost much for a programmer.
[idea] Just give the Game
{a 1 second "Global Cool Down Time" for all buildings}.
and 2 seconds for "Military" buildings.
So you can't spamm wall.
E.g. if you build 4 length wall,
you need to wait for 8 (2x4) seconds
to build a next Military building.