Wasn't sure how to make the dwarves mine. They kinda just crashed out on the gems.
Yeah, they're just really in love with those gems and will walk face first into them. You can direct them to walk past the gem, or put an arrow so they move somewhere else after mining.