Yeah, anytime you fail a roll the idea is that sword takes the brunt of the blow to protect you, and as it does it's score reduces and shards splinter off of it.
I can see the wording there being unclear. The intent is that you gain shards when you fail rolls and take damage or when you break down loot at camp -- not when you successfully attack.