From what I've noticed, it's completely random and fairly rare. Gems also do not spawn immediately at the start of a night. They can show up later on in places you've already checked, so you should be checking and re-checking areas with any downtime that you have.