Alright, I figured out the problem I was having. The timing is extremely sensitive so I have to move the key REALLY fast otherwise it just opens a menu. Not sure why that happens
I must be missing something obvious here... it just keeps saying "Please drag the keys from inside the backpack to the door" but it won't let me drag it, it just opens a menu with "use" (which just repeats the message) and "remove" (which deletes the item)