Cardiovascular disease (CVD) has become one of the most serious diseases that threaten human health. Over the past decades, over 150 million humans have died of CVDs. Hence, timely prediction of CVDs is especially important. Currently, deep learning algorithm-based CVD diagnosis methods are extensively employed, however, most such algorithms can only utilize one-lead ECGs. Hence, the potential information in other-lead ECGs was not utilized. To address this issue, we have developed novel methods for diagnosing arrhythmia. In this work, DL-CCANet and TL-CCANet are proposed to extract abstract discriminating features from dual-lead and three-lead ECGs, respectively. Then, the linear support vector machine specializing in high-dimensional features is used as the classifier model. On the MIT-BIH database, a 95.2% overall accuracy is obtained by detecting 15 types of heartbeats using DL-CCANet. On the INCART database, overall accuracies of 94.01% (II and V1 leads), 93.90% (V1 and V5 leads) and 94.07% (II and V5 leads) are achieved by detecting seven types of heartbeat using DL-CCANet, while TL-CCANet yields a higher overall accuracy of 95.52% using the above three leads. In addition, all of the above experiments are implemented using noisy ECG data. The proposed methods have potential to be applied in the clinic and mobile devices.