They can change even with 50 units of fear. This is a random, which is affected by the total amount of fear available at the moment.