You probably already finished the game by now, but it said that you don't really need to check your inventory for the keys, just interact with the door and you'll be just fine!
I saw a video and i didn't end the game yet... the rainbow key that a book gives in the solar door house, i opened the inventory and suddenly the key disappeared.
