When I get 4 of a kind, there's a delay for when I can grab the baton. (I click, but during the animation, I can't click.) This is not the case for the computer players. I had 4 of a kind, but the computer player grabbed before I could.
Sorry, i forgot to mention in the instructions that you can only grab a baton at the start of your turn. As for the timers, that is part of an anti-frustration mechanic that increases the opponents minimum and maximum response time after you've already won a 1v1 against them. This is to prevent an RNG fest to get the achievements (times get back to normal ranges after you get them all), but i might have overtuned it too much. I will fix this in my next update.