Orcs, cold and hunger – not every settler can endure these trials.
Diseases can spread because of dead people and the living ones will get sick.
To prevent that, you need to build cemeteries – their workers will take care of the dead comrades.