You all are aware that there are XY women and XX men, right? It's common, due to a transposition that relocated the SRY gene (which triggers male growth) to the X and off the Y.
Furthermore, think more in depth: X and Y don't mean squat. It's just that the Y carries the SRY gene, which sets of a developmental sequence leading, usually, to maleness. But development is tricky - mutations and environmental factors can alter it. Studies of Trans individuals have revealed that, whatever their chromosomes, their brains are biologically of the gender they identify with, likely due to developmental anomaly that sent their bodies in one direction, but their brains in another (yes, it's very possible - development involved complex interactions of dozens or hundreds of genes that vary in both time and anatomical space).
So, if an individual, due to developmental anomaly, possessed the body of male but the mind of a female (or vice versa), what are they? Given that the brain is what gives us personality, memory, thought, behavior, and generally everything that makes us what we are, I say the brain is the deciding factor.
Biology is NEVER simple.