Since Mila can fall into her slumber during a wide period of game time, the exact reasoning for the Dark Orbs getting scattered can vary. I think its left to interpretation how the events fall into place.
Also, I don’t recall Ashe appearing in Chapter 3-V. Where was she?!