I love puzzle games, so you've already hit a soft spot in my heart.
I think the hard part about the design you've picked is the fact that I can't see the other sides of the dice when it's sitting on the board.
I know that I could get a physical dice piece, put it front of me, and rotate it that way. I could also try to mentally rotate it, but that's pretty taxing on my feeble 42 year old brain.
As a puzzle it becomes a lot of hard manual work that's outside of the game's intended design space. Instead of focusing on the puzzle, I would need to look at a reference dice, look back at the game, look at a reference, and that's not fun. I want my attention to primarily be on the game in front of me.
I generally feel that way about a lot of these rolling Sokoban style puzzlers, with the exception of Stephen's Sausage Roll. That game gets away with it because the sausage only has two sides, and you can see if a side is burnt from all viewing angles.
If I were to redesign your game, I would take out the dice and replace it with something simpler, like a sausage, something where you can see its entire state from all viewing angles. Then design from there.