No it's actually fine.
And yea, unity physics often bring headache XD
I'm also having problem making it work the way i wanted haha
Or probably you could check for the velocity every time it collide with the enemy before register it as a hit. (the condition for velocity value should be low but you can't use zero. This i guess need a lot of trial and error)