On the very last level, the big guy just sits at the cart and camps, and the dwarves are too stupid to run away.
I don't think it's a bug, just dumb dwarves. When the dwarves get a gem, they make a beeline for the cart. They only stop to punch the small guys. They won't go for food, run away from the big guy, go for bombs, anything. So when they're putting away a gem, they just sit there and take it from the big man.