Deep networks have demonstrated the potential for computer-aided diagnostics for decision support. They have been successful in imaging-based diagnostic tasks for a variety of diseases: neurological conditions such as Alzheimer’s disease, cancers, cardio-pulmonary diseases and many others. Inspired by these successes, this thesis focuses on the diagnosis of thoracic diseases from X-ray images using deep learning. Recently, deep convolutional neural networks have been successfully applied to X-ray data. However, this approach suffers from two drawbacks:(1) classification rates for certain diseases are insufficiently high enough for effective decision support, and (2) the disease features extracted are high-dimensional, limiting visualization of the disease space for knowledge discovery.