I am a great fan of the direction this is going. Personally i feel its very difficult to accurately depict the 'correct' emotion. I feel like every culture and every person would have a different facial expression in a different situation; and the game invalidating my interpretation with a negative sound didn't feel great. The mood was striking and very good; loved the 3d + 2d combo; and lack of distinct details to make me feel a little uneasy.
The essence of what you built is very good; with time, playtesting, and polish, this could tell an interesting story!