Requires some strategy. It doesn't make sense for the goblins to be able to destroy the castle from the other side of the island.
