Warm-3d: A weakly-supervised sim2real domain adaptation framework for roadside monocular 3d object detection

X Zhou, D Fu, W Zimmer, M Liu… - arXiv preprint arXiv …, 2024 - arxiv.org
arXiv preprint arXiv:2407.20818, 2024arxiv.org
Existing roadside perception systems are limited by the absence of publicly available, large-
scale, high-quality 3D datasets. Exploring the use of cost-effective, extensive synthetic
datasets offers a viable solution to tackle this challenge and enhance the performance of
roadside monocular 3D detection. In this study, we introduce the TUMTraf Synthetic Dataset,
offering a diverse and substantial collection of high-quality 3D data to augment scarce real-
world datasets. Besides, we present WARM-3D, a concise yet effective framework to aid the …
Existing roadside perception systems are limited by the absence of publicly available, large-scale, high-quality 3D datasets. Exploring the use of cost-effective, extensive synthetic datasets offers a viable solution to tackle this challenge and enhance the performance of roadside monocular 3D detection. In this study, we introduce the TUMTraf Synthetic Dataset, offering a diverse and substantial collection of high-quality 3D data to augment scarce real-world datasets. Besides, we present WARM-3D, a concise yet effective framework to aid the Sim2Real domain transfer for roadside monocular 3D detection. Our method leverages cheap synthetic datasets and 2D labels from an off-the-shelf 2D detector for weak supervision. We show that WARM-3D significantly enhances performance, achieving a +12.40% increase in mAP 3D over the baseline with only pseudo-2D supervision. With 2D GT as weak labels, WARM-3D even reaches performance close to the Oracle baseline. Moreover, WARM-3D improves the ability of 3D detectors to unseen sample recognition across various real-world environments, highlighting its potential for practical applications.
arxiv.org
以上显示的是最相近的搜索结果。 查看全部搜索结果