Deep neural models, in recent years, have been successful in almost every field, even
solving the most complex problem statements. However, these models are huge in size with …

The remarkable flexibility and adaptability of ensemble methods and deep learning models
have led to the proliferation of their application in bioinformatics research. Traditionally …

In recent years, deep neural networks have been successful in both industry and academia,
especially for computer vision tasks. The great success of deep learning is mainly due to its …

Although convolutional neural network (CNN) paradigms have expanded to transfer
learning and ensemble models from original individual CNN architectures, few studies have …

We formally study how ensemble of deep learning models can improve test accuracy, and
how the superior performance of ensemble can be distilled into a single model using …

Scaling up the convolutional neural network (CNN) size (eg, width, depth, etc.) is known to
effectively improve model accuracy. However, the large model size impedes training on …

Often we wish to transfer representational knowledge from one neural network to another.
Examples include distilling a large network into a smaller one, transferring knowledge from …

Due to the subjective annotation and the inherent inter-class similarity of facial expressions,
one of key challenges in Facial Expression Recognition (FER) is the annotation ambiguity …

Abstract Knowledge distillation is a widely applicable technique for training a student neural
network under the guidance of a trained teacher network. For example, in neural network …

In this paper, we present a thorough evaluation of the efficacy of knowledge distillation and
its dependence on student and teacher architectures. Starting with the observation that more …