You need to build up your physical Stat
After working on the farm, you need to do the demesne storylines to get even higher physical and the other trait that increases when you meditate and answer a question correctly.