There's this perk that increases your lowest stat, however it seems that if that stat has been upgraded with one of the perks that do that (Harder, Stronger, Faster), the game instead of giving it +1 gives it -1 (in my case my Agility went from 4 to 3).
Aside from that, game looks really promising and I've been thoroughly enjoying it!
Btw, I'm using version 0.1.25