Of course, building several builders is quite an obvious way out. But I am not talking about the fact that it is difficult for the player to solve the problem that arises, but again about the same thing. On the counterintuitiveness of mechanics. After all, when creating the game, you probably reasoned traditionally. The building cannot work when a team of gloomy men with hammers and saws runs into it. And in most cases it is. Sawdust falling into the dough interferes with the baker, and the constant hammering above his head interferes with the accountant. But who, tell me, in the builders' premises can these same builders interfere with? :) Their work cannot interfere with their work, it becomes obvious if you take the trouble to think about it. I will not dispute the 10% efficiency you have now introduced, but personally I would give at least 80%. And that's not enough, but you need to depict at least some kind of fine. For, let's say, the general chaos caused by their work. After all, in fact, the builders' office is a warehouse and recreation facilities for employees. Of course, without the opportunity to drink beer and sleep for six hundred minutes after lunch, the work will slow down somewhat. But they must have a foreman for this, who, with the help of the magic power of the word, should increase their efficiency to average values :)
However, I digress again. I think you've got the gist of it. Builders working in their own office simply have no one there to interfere.