Funny... not sure if supposed to be part of the experience or not but it would appear that the only way to delete the file and close some kind of background program(?) after extracting is to restart your computer. Kind of annoying but if it floats your boat that's kinda cool.

Which OS are you running?

I had the same issue (I'm using Windows), but I didn't have to restart the computer (I just ended the process instead). It seems that after completing the story, the application window shut down, but not the process itself.