Oh shoot, I never saw the 2nd key. Thought I saw everything in the area. Gonna have to look again and get me some bread.
*edit* found it! Yeah, maybe if you color code the keys (like red key for the red gate) it would help. Otherwise you are like wait didn't I already use that key?