Babies are at special risk before birth and during the first few years of life. During this time, the brain grows a lot. Exposure to certain diseases, poor nutrition, or lack of oxygen can sometimes cause epilepsy.