A long time ago in the machine learning literature, the idea of incorporating a mechanism
inspired by the human visual system into neural networks was introduced. This idea is …

Transformers based on the attention mechanism have achieved impressive success in
various areas. However, the attention mechanism has a quadratic complexity, significantly …

Recently, a new line of works has emerged to understand and improve self-attention in
Transformers by treating it as a kernel machine. However, existing works apply the methods …

Recent legislation has led to interest in machine unlearning, ie, removing specific training
samples from a predictive model as if they never existed in the training dataset. Unlearning …

Rich user behavior data has been proven to be of great value for Click-Through Rate (CTR)
prediction applications, especially in industrial recommender, search, or advertising …

Cued Speech (CS) is a pure visual coding method used by hearing-impaired people that
combines lip reading with several specific hand shapes to make the spoken language …

Transformers have emerged as a preferred model for many tasks in natural langugage
processing and vision. Recent efforts on training and deploying Transformers more …

Autoregressive Large Language Models (LLMs) have achieved impressive performance in
language tasks but face two significant bottlenecks:(1) quadratic complexity in the attention …

While GPU clusters are the de facto choice for training large deep neural network (DNN)
models today, several reasons including ease of workflow, security and cost have led to …

Transformers are central in modern natural language processing and computer vision
applications. Despite recent works devoted to reducing the quadratic cost of such models …