In recent years the pavement engineering community has shown increasing interest in shifting from a stationary falling weight deflectometer (FWD) to moving testing platforms such as the traffic speed deflectometer (TSD). This paper dealt with comparing TSD measurements against FWD measurements; it focused on the comparison methodology, utilizing experimental data for demonstration. To better account for differences in loading conditions between the two devices a new FWD deflection index was formulated first. This index served as reference/benchmark for assessing the corresponding TSD measurements. Next, a Taylor diagram was proposed for visualizing several comparison statistics. Finally, a modern agreement metric was identified and applied for ranking comparison results across different datasets. Overall, the suggested methodology is deemed generic and highly applicable to future situations, especially for assessing the worth of emerging device upgrades or improved interpretation schemes (or both).