Although Transformer-based language representations achieve state-of-the-art accuracy on
various natural language processing (NLP) tasks, the large model size has been …

Recently, Transformers gradually gain popularity and perform outstanding for many Natural
Language Processing (NLP) tasks. However, Transformers suffer from heavy computation …

Transformers are considered one of the most important deep learning models since 2018, in
part because it establishes state-of-the-art (SOTA) records and could potentially replace …

BERT is the most recent Transformer-based model that achieves state-of-the-art
performance in various NLP tasks. In this paper, we investigate the hardware acceleration of …

Transformer-based deep learning models have become a ubiquitous vehicle to drive a
variety of Natural Language Processing (NLP) related tasks beyond their accuracy ceiling …

Neural networks based on Long Short-Term Memory (LSTM) are widely deployed in latency-
sensitive language and speech applications. To speed up LSTM inference, previous …

In natural language processing (NLP), the" Transformer" architecture was proposed as the
first transduction model replying entirely on self-attention mechanisms without using …

Recently, pre-trained Transformer [1] based language models such as BERT [2] and GPT [3],
have shown great improvement in many Natural Language Processing (NLP) tasks …

Designing hardware accelerators for deep neural networks (DNNs) has been much desired.
Nonetheless, most of these existing accelerators are built for either convolutional neural …

Transformer has emerged as a popular deep neural network (DNN) model for Neural
Language Processing (NLP) applications and demonstrated excellent performance in …