In boys, hormones made in the testicles usually keep breast tissue from growing too much. When boys and men have gynecomastia, female hormones cause the breast tissue to swell. This causes breasts that are larger than normal for men. Gynecomastia does not cause excess fat in the breasts.