You're not even just limited to D6. I own a Halfling Haversack with 20 full sets of tiny polyhedral dice and, even though I have a very minor tremor, I can stack 20 tiny D6 almost without a problem. But if I try to use a mix of D6, D12 and D20 like in The Collapse, managing to stack even just 10 dice becomes a chore.