Self defense is not about weight classes. This isn't boxing. It's about repelling an immediate physical attack.
Both women go after the guy, attacking him. Again, gender does not bestow a juridical privilege upon a certain group of people regarding infliction of physical pain and bodily harm. It also does not dictate its tolerance under forfeiture of basic human right to another group.
Had one of the women pulled a knife and stabbed the guy while standing behind him, his size and strength would have been of no consequence for the outcome of this fight. If a person is being outnumbered by the attackers, any self defense actions have to be as swift and efficient as possible.
That's exactly what is shown in that video sequence. The guy only punched his aggressors enough to stop the attack before walking away, without any unnecessary retaliation.
Ask a traumatologist about the possible severity of cranial and facial injuries caused violent women or adolescents. It includes everything from fractured teeth, noses, cheek and orbital bones to eye and ear lesions, all the way to intracranial hematomata. I remember a case from my circle of acquaintances where a mother, a doctor, blinded her teenage son on one eye by striking him in the face.
And why are other men, solely because of their gender, obliged to risk their lives as well as their physical and legal integrity in support of the two aggressors? Inform the authorities in charge and let them sort it out.
Equality between men and women necessitates responsible behavior from both sides. Humans are not supposed to physically attack other humans, that's what the law says.
Having said that, I commend and fully comprehend your chivalrous attitude. But these two women don't seem to be the kind of ladies that deserve our pity and gallant self-sacrifice.