Probabilistic inversions of wide angle reflection and refraction data for crustal velocity models are regularly employed to understand the robustness of velocity models that can be inferred from these data. It is well understood that the uncertainties associated with the picks of individual arrivals contribute to overall model uncertainty. Typically only a modicum of effort is devoted to quantifying uncertainty in the traveltime picks; a constant noise estimate is commonly assigned to a given class of arrivals. Further, determining the class of arrivals is often left to the behest of the interpreter, contributing additional uncertainty to the data that is both difficult to quantify and may be altogether incorrect. Given the crucial role data uncertainty plays in characterising model robustness, there is a need to thoroughly and appropriately quantify uncertainty in the traveltime data which itself is inferred from the waveform. Here we propose a method that treats arrival or phase classification as part of the velocity model building (inversion) framework using the well-established expectation-maximization (EM) algorithm.