You can pick them along the way and drop them close to ghost room.
i should do that yeah or at least cleanup all the unneeded stuff in the ghost room before starting the banishing process.