Deep learning is nowadays a buzzword and is considered a new era of machine learning which trains the computers in finding the pattern from a massive amount of data. It mainly describes the learning at multiple levels of representation which helps to make sense on the data consisting of text, sound and images. Many organizations are using a type of deep learning known as a convolutional neural network to deal with the objects in a video sequence. Deep Convolution Neural Networks (CNNs) have proved to be impressive in terms of performance for detecting the objects, classification of images and semantic segmentation. Object detection is defined as a combination of classification and localization. Face detection is one of the most challenging problems of pattern recognition. Various face related applications like face verification, facial recognition, clustering of face etc. are a part of face detection. Effective training needs to be carried out for detection and recognition. The accuracy in face detection using the traditional approach did not yield a good result. This paper focuses on improving the accuracy of detecting the face using the model of deep learning. YOLO (You only look once), a popular deep learning library is used to implement the proposed work. The paper compares the accuracy of detecting the face in an efficient manner with respect to the traditional approach. The proposed model uses the convolutional neural network as an approach of deep learning for detecting faces from videos. The FDDB dataset is used for training and testing of our model. A model is fine-tuned on various performance parameters and the best suitable values are taken into consideration. It is also compared the execution of training time and the performance of the model on two different GPUs.