This mechanic is a super great idea. Juicing up the feedback and goal mechanics and plop this in the middle of a narrative and you have yourself a super awesome gem.
I like the idea of like a system of faces that you learn over time and slowly add the complexity and open up more options and interactions with characters. As you memorize things you start to feel super badass. Then each new challenge could throw it's own wrench in the socket.
By the by, the 3rd level emoji don't seem to give me the green. The green feedback that this is working is a great idea, by the way. And the leniency within the time-sensitive environment keep me trying even if I feel overwhelmed.