Well, exactly, these are differences on average. You can't look at a brain & say this one belongs to a man, this one a woman. We're not there yet. There is no brain possessed by a woman that would rule her out as being correctly identified as a woman.
I'd have to read more of the science to have a worthwhile view about this matter. If it turns out there is a "trans brain" (shorthand, obv.), then that's an interesting result. But orthogonal to the issues surrounding trans activism (which I'm not sure we disagree about anyway).