No. It shouldn't be affected by anything else. After the talk with the option to encourage him or discourage him the option to improve defense should be available in the sawmill before hour 15. Not in the talk menu, just before the option to leave the sawmill.