Theres a somewhat well hidden enterance to a cave at the left bottom corner of route 1 and when you go in and interact with the stars you will be presented with a minigame in which you can get all sorts of things and one of those things are gems which also includes normal gems and im pretty sure even bug gems. If you didnt get lucky on one of your first tries then these minigames respawn every 3 days or so meaning you can get a basically infinite amount of them.