Chronic pulmonary disease is one of the leading causes of mortality in the United States. Continuous passive monitoring of subjects using mobile sensors can help detect disease, estimate severity, track progression over time, and predict adverse exacerbation events. One of the most convenient avenues to realize this goal is through analysis of passively recorded natural speech patterns. It has been previously established that diseases such as asthma and chronic obstructive pulmonary disease (COPD) affect pause patterns and prosodic features of speech. In this study we present an exploration of the feasibility of using speech features from natural speech to detect pulmonary disease. Experiments were conducted on a cohort of 131 subjects: 91 with asthma and/or COPD, and 40 healthy controls. Patients and healthy subjects were differentiable with 68% accuracy; moreover, the subset of patients with the highest disease severity were detected with 89% accuracy.