You might be just using the 'skip' feature without knowing it.
Altho in the past it didn't affect the timer. Something may have changed with all the Renpy updates. I'll look into it tomorrow.