Convolutional Neural Network (CNN) models are a type of deep learning architecture introduced to achieve the correct classification of breast cancer. This paper has a two-fold purpose. The first aim is to investigate the various deep learning models in classifying breast cancer histopathology images. This study identified the most accurate models in terms of the binary, four, and eight classifications of breast cancer histopathology image databases. The different accuracy scores obtained for the deep learning models on the same database showed that other factors such as pre-processing, data augmentation, and transfer learning methods can impact the ability of the models to achieve higher accuracy. The second purpose of our manuscript is to investigate the latest models that have no or limited examination done in previous studies. The models like ResNeXt, Dual Path Net, SENet, and NASNet had been identified with the most cutting-edge results for the ImageNet database. These models were examined for the binary, and eight classifications on BreakHis, a breast cancer histopathology image database. Furthermore, the BACH database was used to investigate these models for four classifications. Then, these models were compared with the previous studies to find and propose the most state-of-the-art models for each classification. Since the Inception-ResNet-V2 architecture achieved the best results for binary and eight classifications, we have examined this model in our study as well to provide a better comparison result. In short, this paper provides an extensive evaluation and discussion about the experimental settings for each study that had been conducted on the breast cancer histopathology images.