Sex is a biological fact. It is not socially constructed. "Gender" is also only partially a social construct as there are behaviours and characteristics that are overwhelmingly shown by only males or females. These behaviours and characteristics are based on biological sex. For example, some women are tall but most women are short compared to men and this is not caused by society but by genes and sexual reproduction. Same goes for things like less muscle mass in women or men balding with age. Women are on average more shy and more submissive than men because being aggressive and trying to get into fights with other people is not a good survival strategy for a short, pregnant woman. Some of these women can still reproduce of course, because they can counterbalance their "shortcomings" with other things - just like every person has shortcomings and is not the perfect mate for the opposite sex. These are not social constructs but biological facts and they are the main cause for "gender stereotypes". Gender stereotypes are based on reality, they are just sometimes cranked up to eleven and of course don't include the minority of people who do not express the stereotypical behaviour - and if we discriminate these people, that (of course) is bad.
There are hardly ever absolute categories in biology but that doesn't mean we can't put labels on things or sort things into categories. Humans are a species with two legs and two arms. That's how it is. There may be an incredibly small number of people with only one arme or three legs but they are very rare deviations from the norm.
Also, I hate it when people bring up the "gender spectrum". Gender is not a spectrum! 49.9% male, 49.9% female and 0.2% intersex is not a spectrum! These are two distinct categories, with expected biological hiccups in-between. Because biology isn't perfect.
Lastly, biological sex does not change if your shoulders are too broad / narrow or you are short or tall! A tall woman is 100% a woman, no spectrum here at all. A muscular woman is still 100% a woman. A short man is still a man, 100%. A man with a small penis is still 100% a man. A woman who is infertile is still 100% a woman. A submissive man is still a man - yeah, 100%. Who keeps making this up? People may say a muscular woman is an unattractive woman. Or that a short man with a small penis is unattractive. But that doesn't mean someone is not a man or a woman, just that they are not sexually attractive! And sexual selection only works because gender is binary. Our very existence is based on the fact that gender is binary because we all have been sexually selected for! Also, what men find attractive in women and vice versa is not socially constructed. It is based on the biological reality that weak men can't provide for a pregnant female and a woman with small hips cannot give birth to a baby (to give just two examples). Sexual dimorphism is real and it is not socially constructed.
