Diseases that affect the hypothalamus and the pituitary glands can cause low testosterone. These include cancer, inflammation and autoimmune diseases of either gland.