I've beaten this after a day or so of playing it too much. It's really cool! My favorite part is the board itself, and I definitely think you could do more with it - enemies that block slots of the board, Lonely Dice that need to not be next to anything else in order to work, alternate characters with different boards, upgrades to particular spaces on the board, who knows!
There also isn't that much... dice-like about the dice, I guess. Nothing rolls them, they're just static numbers. Maybe a Boost-like dice type that rerolls the things they're next to? (With some kind of cap, of course).
I also think the rarity system needs to be more obvious in-game; there were a lot of times I was confused about why my silver Block die was only worth 3 points.