I definitely had this happen in the lower numbers, on one of my first few runs the count up didn't happen at all until I hit zero. I wasn't going quick either, the timers definitely ran out. I was really surprised when it went back up to 1 on that run because I hadn't seen that yet and assumed it was a sort of "final boss" of the game, but then in my subsequent runs the same didn't always happen! No idea what I was doing that broke it.
Still had a ton of fun with it though, just letting you know in case you plan to work on this again after the jam!