The proposed research work develops a system to enhance the performance of university students and lecturers by providing an excellent statistical insight. Already existing research works have attempted to solve independent classroom challenges that are related to measuring the student attention and marking student attendance but the existing research works have not combined theimportant aspects into one system. Hence, the proposed research wor has been carried out on various main aspects such as attendance register, monitoring student behavior as well as lecturer performance and lecture summarization. The system will incorporate tools and technologies in the different domains of artificial intelligence, machine learning, and natural language processing. After implementing and testing the proposed method it has been concluded that the student activity recognition process has been performed much better than the other emotion and gaze components by providing 94.5% results. The proposed system can determine the lecturer's physical activities and the quality of the lecture content with a reasonable accuracy. The summarized lecture has showed 70% similarity to actual lecture content and student attendance by using Face Recognition was marked with 83% accuracy. This research concludes that the automation of major classroom activities will impact the students and lecturers positively. Also, this system yields valuable results and increases the productivity of higher education institutions in the future.