With the popularity of autonomous vehicles and the rapid development of intelligent transportation, the application scenarios for detecting pedestrians in everyday life are becoming more and more widespread, with high and high application value. Pedestrian detection is the basis of many human-based tasks, including speed tracking, pedestrian motion detection, automatic pedestrian recognition, and appropriate response measures, or rejecting true false pedestrian detection. Various researchers have done a lot of research in this area, but there are still many errors in the correct identification of rejecting true false pedestrians. This article focuses on the design and implementation of real pedestrian discovery using deep learning technology to identify pedestrian rejections. In this work, our goal is to estimate the achievement of the current 2D detection system with a 3D Convolutional Neural Network on the issues of rejecting true false pedestrians using images obtained from the car’s on-board cameras and light detection and ranging (LiDAR) sensors. We evaluate the single-phase (YOLOv3 models) and two-phase (Faster R-CNN) deep learning meta-structure under distinct image resolutions and attribute extractors (MobileNet). To resolve this issue, it is urge to apply a data augmentation approach to improve the execution of the framework. To observe the performance, the implemented methods are applied to recent datasets. The experimental assessment shows that the proposed method/algorithm enhances the accuracy of detection of true and false pedestrians, and still undergoes the real-time demands.