I haven't done any projects that include super detailed audio analysis so I don't have the exact knowledge. I used the same technique for a flappy bird game, but you were jumping by clapping to microphone :)
One another idea that could be useful is implementing a rhythm system. I am pretty sure you can develop a metronome for your game. And having that might help with timing your audio inputs in a way. Or it might give you some other ideas for the game.
I tried developing one in this game, it is not done but maybe you can understand what I mean by a metronome.
(https://ahmetahaydemir.itch.io/magecore)
Be sure to update your game on itch if you ever find the chance, I would love to see it!