Using image sentiment analysis rather than text sentiment analysis to make different judgments has grown increasingly popular recently. In recent years, transfer learning techniques have been employed extensively in developing end-to-end image sentiment analysis methodologies, which is expected to continue. Deep learning algorithms have produced outstanding outcomes in a variety of applications. Even though image-based sentiment analysis is tricky, there seems to be much space for progress. Research presented here suggests that a VGG-19-based approach, which can easily be used to concentrate on vast body areas, such as the face, offers a significant gain over earlier studies. This research aims to improve image categorization performance by using the well-known deep convolutional neural network, VGG19, and other deep features. Emotion detection and classification with diverse emotions utilizing a VGG-19 (CNN-based) architecture can be easily performed using CK+, FER2013, and JAFFE datasets. The results show that the proposed method can reach accuracy up to 99%. It can study a person's emotional habits and psychological condition using the approach.