The Threat Roll resolves the consequences of the threat, not the Scoundrel's own effect. The Scoundrel has whatever effect that's been established, regardless of the Threat Roll outcome (unless you include the threat of failure).
So, On a 6, you avoid the threat. That means it has no effect on you.
On a 4/5, the consequence is reduced. So in your example, level 4 Harm would be reduced to level 3.