I thought the same as well. Turns out you have to press c with the bread in your inventory. (to set the bread to outgoing I guess?) Then he will accept the bread and you get a victory screen
