That's probably because that's who you argue with most. I get it from them but also from feminists quite a lot because I argue with them too. Only when all men are suggested to be at fault. This is a huge problem right now which does need addressing.
In the same way you'd like to see me not associate w/ people who say 'slut' 'redpill' and 'alpha/beta' (and I mock those often), I'd like to see you not associate with the rhetoric of 'mansplaining' and #NotAllMen = bad. I think this undermines your credibility re gender equality