Heart rate variability (HRV) is a widely used measure for the variation of the heartbeat intervals controlled by the autonomic nervous system (ANS), which is primarily obtained by electrocardiogram (ECG) signals. In general, HRV of a healthy person exhibits long-range correlations with dynamic fluctuations, whether the complexity of HRV decreases with aging and incidence of disease. Recently, the complexity of differential inter-beat intervals, referred to differential R-R intervals, is known to be more effective than original R-R intervals to reflect HRV. The multiscale based entropy methods have been developed to quantify HRV using R-R intervals. In spite of their capability, it still remains unreliable quantification of HRV. Here, we propose a new multiscale complexity quantification measure with differential R-R intervals for HRV analysis. To verify the performance of the proposed method, we evaluate the complexity of differential HRV extracted from ECG signals of congestive heart failure (CHF) patients and healthy subjects. The results show that multiscale distribution entropy (MDE) of differential R-R interval has improved capability for quantifying the complexity of HRV regardless of the length of time series.