Yes, if you dump Strength, or somehow have negative endurance it would subtract it from the 10 base HP.
So your example of -4 Endurance would make stating HP to 6.
Another example if you have -1 Strength, your HP would be 9 instead of 10.
Thanks for trying out my game, hope you enjoy it.
Feel free to comment if you have any more questions!