I think each round reset your stats. The dice merely modify it for that round. Dice 1 is the number, Dice 2 is the modify type (One of three stat, each with a + or a -). So a 2 Glasses (with a minus) is -2 to INT score that round. So for your example, 4 didn't lose to glasses. It was "4" & "INT minus". The +/- symbol are quite small.The reset probably exists to balance it out, so a few minuses don't permanently ruin your run.
You then need to try to match the dialogue option to the stat it represents, if your stat is high enough it wins the round. Win enough rounds to reach $30 and you win by paying for her meal.