When trying to use chroma key with videos that weren't made for it, I can often set it well enough to remove most of the background.  However, there are many times when I can't quite get it to work without also removing people's eyes and teeth, which is pretty off-putting to say the least.  I'm wondering if it would be possible to use some basic face-tracking to exclude them from the chroma key removal.  It wouldn't need to be super detailed, as long as the eyes and mouth are preserved, that would be enough to work with.  

Of course, I don't really know what it would take to make that work, it's just a thought I had.  Feel free to ignore this if it's not feasible.