I'm sure we can investigate genetic variations and hormone levels but these require blood tests: you can't assess these things just by looking as far as I know.
Simply looking at what's situated between the legs is far more convenient and pivotal in distinguishing between male or female. I'm sure we can look at other things but our genitals will never not be in the conversation.
The breast area can also show cromosome variations, as well as the testicles. I'm no expert but here is a source among probably many others: