Very good idea with a lot of potential. I think it's a little too punishing though. I have a good chance of dying the first time I let the enemies get a turn where I'm too close. Another thing I would change is the distinction between the dice. The number is not enough I feel, maybe if you coloured them according to their max value, they would be easier to differentiate at a glance.
I'm actually a little sorry you didn't get to flesh it out more. It really has a lot of potential. Good work!