When a male and a female create a baby, they each give 1 sex chromosome. If the man gives a Y, the child is a boy. The woman can only give an X. So, the man's chromosome determines the sex of the baby.