Automating the scoring of Whole-Slide Images (WSIs) is a challenging task because the search space for selecting region of interest (ROI) is huge due to the very large sizes of WSIs. A Multifaceted Fused-CNN (MF-CNN) and a Hybrid-Descriptor are proposed to develop an integrated scoring system for Breast Cancer histopathology WSIs. Suitable color and textural features are identified to help mitotic count based selection of ROIs at lower resolution. To recognize complex patterns, the MF-CNN considers multiple facets of the input image. It counts mitoses, extracts handcrafted features from ROIs and utilizes global texture of the images to form a Hybrid-Descriptor for training a classifier assigning scores to WSIs. The proposed system is evaluated on a publicly available benchmark (TUPAC16) and produced the highest score of 0.582 in terms of Cohen’s Kappa. It surpassed human experts’ level accuracy of ROI selection and can therefore reduce the burden of manual ROI selection for WSIs.