Chronic diseases are growing exponentially. Today, there are over 900 million individuals suffering with some chronic diseases around the world. For this reason, e-health systems are being developed to design a better quality of life for patients. For instance, we have systems for detection of epilepsy, monitoring of vital signs, control of diabetes, among others. Deep learning has being an important technique embedded in these systems to predict and sort the data without the need of a 24-hour monitoring specialist. However, by combining e-health systems and deep learning techniques also brings several challenges that need to be overcome. Based on this context, this work-in-progress proposes an e-health system based on fog and cloud computing, using deep learning to predict epileptic seizures. We also present some research challenges for this implementation.