Yes! I thought the same! Something like only the outline or with a certain alpha transparency. Or, if you are going to change the icons to match the gems, than the placeholders could be "blank", with no details, and than the gems would fit and have color + details! In my opinion that also would work well.
Best of luck!