In this work, we propose a two-stage classifier based on the analysis of the heart sound's complexity for murmur identification and classification. The first stage of the classifier verifies if the heart sound (HS) exhibits murmurs. To this end, the chaotic nature of the signal is assessed using the Lyapunov exponents (LEs). The second stage of the method is devoted to the classification of the type of murmur. In opposition to current state of the art methods for murmur classification, a reduced set of features is proposed. This set includes both well-known as well as new features designed to capture the morphological and the chaotic nature of murmurs. The classification scheme is evaluated with three classification methods: Learning Vector Quantization, Gaussian Mixture Models and Support Vector Machines. The achieved results are comparable to reported results in literature, while relying on a significant smaller set of features.