Yes, that's how it should behave. The orbs are not adding to what you have in previous round. When you start new round, orbs are reset to 0. So if your round is bad, you can get less than you had last time.
