Rice is an essential staple food for human nutrition. Rice varieties worldwide have been planted, imported, and exported. During production and trading, different types of rice can be mixed. Due to rice impurities, rice importers and exporters may lose trust in each other, requiring the development of a rice variety identification system. India is a significant player in the global rice market, and this extensive study delves into the importance of rice there. The study uses state-of-the-art deep learning and TL classifiers to tackle the problems of rice variety detection. An enormous dataset consisting of more than 600,000 rice photographs divided into 22 different classes is presented in the study to improve classification accuracy. With a training accuracy of 96% and a testing accuracy of 80.5%, ResNet50 stands well among other deep learning models compared by the authors. These models include CNN, Deep CNN, AlexNet2, Xception, Inception V3, DenseNet121, and ResNet50. Finding the best classifiers to identify varieties accurately is crucial, and this work highlights their possible uses in rice seed production. This paper lays the groundwork for future research on image-based rice categorization by suggesting areas for development and investigating ensemble strategies to improve performance.