He should be able to give it to you. Perhaps try mentioning "key to the shed" or "shed key" explicitly.
Will be sure to try this later.
It worked finally, had to use "key to the shed" to get the right key.