Infections can affect the amount of sperm made or sperm health. Some also cause scarring that blocks the passage of sperm. Examples include: Swollen testicles due to mumps. Swollen prostate. Sexually transmitted diseases, or STDs, like chlamydia and gonorrhea.