The general idea is that every time you want to place a tile, you draw randomly from the pile and must try to place it, so you never know if it will be possible (the only tile you know beforehand is the one from the Unforgettable Memory). Maybe it isn't very clear in the document, I'll try to make it clearer.
Regarding the actual numbers, in this edition they are not really relevant. They could be used for oracles (and that would enable solo-play), could be used to transform the final tile disposition into a map or a dungeon (if you associate every number to something, like in Tiny Library's dice-based dungeon generator) or to create some alternate rules. These are all options I'm considering if I expand this game at a later point, without the 2-page restriction.