I figured it out. You really gotta spec into the number of units you can summon. Normal upgrades alone won't get you far.
