The idea is cool, but the execution needs some work - collisions felt a bit off for example. Another suggestion is the punching animation shouldn't be interruptable if damage is delivered at the end of the punch; either speed up the animation or make the player commit to an attack, it will feel more impactful that way.
A little bit of polish and you've got a gem here. Was this inspired by Crank?