Sparsely activated neural networks with conditional computation learn to route their inputs
through different" expert" subnetworks, providing a form of modularity that densely activated …

Abstract Video Motion Magnification (VMM) aims to reveal subtle and imperceptible motion
information of objects in the macroscopic world. Prior methods directly model the motion …

Large Language Models (LLMs) have achieved state-of-the-art performance across various
language tasks but pose challenges for practical deployment due to their substantial …

Abstract Sparsely Mixture of Experts (MoE) has received great interest due to its promising
scaling capability with affordable computational overhead. MoE models convert dense …

Transformer based large language models have achieved tremendous success. However,
the significant memory and computational costs incurred during the inference process make …

In recent years, large-scale models can be easily scaled to trillions of parameters with
sparsely activated mixture-of-experts (MoE), which significantly improves the model quality …

In recent years, the Mixture-of-Experts (MoE) technique has gained widespread popularity
as a means to scale pre-trained models to exceptionally large sizes. Dynamic activation of …

The development of neural techniques has opened up new avenues for research in
machine translation. Today, neural machine translation (NMT) systems can leverage highly …

Mixture-of-experts (MoE) architecture has been proven a powerful method for diverse tasks
in training deep models in many applications. However, current MoE implementations are …

Mixture-of-experts (MoE) models that employ sparse activation have demonstrated
effectiveness in significantly increasing the number of parameters while maintaining low …