The detection of seat belts is an essential aspect of vehicle safety. It is crucial in providing protection in the event of an accident. Seat belt detection devices are installed into many automobiles, although they may be easily manipulated or disregarded. As a result, the existing approaches and algorithms for seat belt detection are insufficient. Using various external methods and algorithms, it is required to determine if the seat belt is fastened or not. This paper proposes an approach to identify seat belt fastness using the concepts of image processing and deep learning. Our proposed approach can be deployed in any organizational setup to aid the concerned authorities in identifying whether or not the drivers of the vehicles passing through the entrance have buckled their seat belts up. If a seat belt is not detected in a vehicle, the number plate recognition module records the vehicle number. The concerned authorities might use this record to take further necessary actions. This way, the organization authorities can keep track of all the vehicles entering the premises and ensure that all drivers/shotgun seat passengers are wearing seat belts.