You'll find that men tend to have lots of physical advantages over women, ranging from (a lot) more natural muscle tissue to just tending to be taller and lots of other obvious stuff we all know good and well just from living in the world. These characteristics provide men with, on average, real and meaningful advantages in both strength and speed. I mean yeah, there are some exceptions, some men who do nothing to develop and hone any of that, and some women who work their asses off to build themselves up, but this is the rule and it's clear enough for all of us to be able to readily observe it. There is no amount of gender identification or artificial testosterone reduction that's ever plausibly going to truly level nature's fundamentally unequal playing field. When men (by which here I mean biological males) participate in women's sport, they tend to have a natural and unfair advantage. That's just a fact. This becomes especially significant in contact sports like American football or rugby or mixed martial arts, for example, where the difference between single-sex and unisex competitions could be measured not just in trophy allocations, but in broken bones.
Conversely, when women participate in men's sports, they enjoy the physical disadvantages of being female. That is why you don't hear of transmen setting new records in men's sport. Gender identity ideology contains no advantages for women (biological females). It confers social benefits only to men (biological males).
Men who set records in women's sport, I think you'll find, tend to achieve times/weightlifting accomplishments/whatever applies that would only make them average athletes in the corresponding men's leagues. They don't tend to be objectively great athletes, in other words. They're just biologically male and have that natural advantage in women's athletics. The fact that they're rarely great athletes by the standards of their sex could be part of their motivation for choosing to compete in the women's league, in fact.
As far as I know, a man has never set a record in women's sports. A trans woman who is competing in female sports is not going to be putting up the numbers of elite males because as far as I'm aware, any major sports league requires at least some period of hormone therapy before one is able to compete. To the extent that they may retain an advantage, it would be incredibly unlikely for them to retain enough that they could still compete in the men's leagues. Judging a trans woman who has gone through HRT by comparing her to a man who has not undergone HRT is kind of ludicrous.