in fact, not punching a nazi when you have the opportunity is in itself a net negative, so we can say it’s immoral to not punch a nazi whenever possible.

Punching Nazis is always self defence, since being a Nazi in public is an act of violence.