There's a secret path above one of the squares that a cave crawler spawns in, it takes you to a room where the Emerine golems are
I don't know if perception plays a part in this but I have 15 just in case, and also seems like a chance thing so going back and fort on this square may help. Hope this helps you.