The main idea is pretty fun! I had a few turns in a row where I got a one, and that was pretty less-than-ideal from a gameplay perspective. Even if it's not FAIR I would suggest adding some logic skew more 5's and 6's after 1's and 2's to make the player feel lucky :)