Fault diagnosis of rotating machinery is crucial to improve safety, enhance reliability and reduce maintenance cost. The manual feature extraction and selection of traditional fault diagnosis methods depend on signal processing skills and expert experience, which is labor-intensive and time-consuming. As a typical intelligent fault diagnosis method, the convolutional neural network automatically learns features from original data, but it is extremely difficult to design and train a deep network architecture. This paper proposes a fault diagnosis scheme combined of hierarchical symbolic analysis (HSA) and convolutional neural network (CNN), which achieves laborsaving and timesaving preliminary feature extraction and accomplishes automatically feature learning with simplified network architecture. Firstly, hierarchical symbolic analysis is employed to extract features from original signals. The extracted features are able to identify different health conditions under various operating conditions. Then, convolutional neural network instead of human labor is used to learn the complex non-linear relationship between features and health conditions automatically. The architecture of CNN diagnosis model is simple and convenient to implement. Finally, a centrifugal pump dataset and a motor bearing dataset are adopted to validate the effectiveness of the proposed method. The diagnosis results show that the proposed method exhibits superior performance compared with shallow methods and deep learning methods.