What is the model you are using? If you can send it over to me it would help a lot so I can try to see what it is
here i send u the .blend and the .glb so you can try https://gofile.io/d/RHGH4d
I looked at the glb in Blender and this model might exceed the vertex limit (which is around 17,000) for SMD files which is why it might be failing to compile (this model is 85,000 vertices). This is not a limitation of the program though, it's a limitation of the source engine which is a pretty old engine that garry's mod runs on and it isn't meant to handle models that are very detailed. Although I might make it so highly detailed models will work with a different solution though