I had 14 books, 4 in my hand with 25 seconds left, but my momentum hit the teacher and I was done. That was a close one!
It's good that there is a penalty when you hit the wall, but it forces you to wait, which slows down the tempo a bit.
How about simply having a 5-second penalty that is immediately deducted from the remaining time?
Another idea would be to have the books you are holding scattered around you and you would have to pick them up.