Exactly! Depending on the difficulty of the action in question, the Narrator can choose to give Advantage or Disadvantage, and still choose the "degree" of this, choosing to double the d6 or d4. It's a mechanic I implemented in my other project, which also uses dice with different sided numbers, but there it's a Triad (d4 + d6 + d10) and uses the same idea of gradation of advantage and disadvantage.
Remembering that, since we're rolling below here, when using Advantage, you'll choose the lowest die, when using Disadvantage, you'll choose the highest.