You can reach the orange flower orb of key fire with *zero* uses of key after you pick key up. You should work backwards from that logic: "If a path forces me to use key, then this must not be the way." You also will need to practice your backtracking walk to avoid setting off key by accident at several points.
Hint: Lbh arrq gb hfr sver orsber lbh teno xrl gb pyrne fbzr fcnpr