Familiar 1: You need to have discovered every room in the game to get this one.
Familiar 20: Obtained by 100% the game's achievements.
Which achievements are you missing (like, which other achievements are around them)? It sounds like you're on the cusp.