Large foundation models, including large language models (LLMs), vision transformers
(ViTs), diffusion, and LLM-based multimodal models, are revolutionizing the entire machine …

Large Language Models (LLMs) represent a class of deep learning models adept at
understanding natural language and generating coherent responses to various prompts or …

Applying Reinforcement Learning (RL) to sequence generation models enables the direct
optimization of long-term rewards (\textit {eg,} BLEU and human feedback), but typically …

Neural architecture search (NAS) has made tremendous progress in the automatic design of
effective neural network structures but suffers from a heavy computational burden. One-shot …

Recently, numerous efficient Transformers have been proposed to reduce the quadratic
computational complexity of standard Transformers caused by the Softmax attention …

Transformers have dominated empirical machine learning models of natural language
processing. In this paper, we introduce basic concepts of Transformers and present key …

The Transformer is an extremely powerful and prominent deep learning architecture. In this
work, we challenge the commonly held belief in deep learning that going deeper is better …

Large language models achieve state-of-the-art performance on sequence generation
evaluation, but typically have a large number of parameters. This is a computational …

Defect recognition is an essential aspect of intelligent manufacturing, but it is a challenging
task with noise and unpredictable uncertainties, where convolutional neural networks …

Given the wide and ever growing range of different efficient Transformer attention
mechanisms, it is important to identify which attention is most effective when given a task. In …