An overview of energy-efficient hardware accelerators for on-device deep-neural-network training

J Lee, HJ Yoo - IEEE Open Journal of the Solid-State Circuits …, 2021 - ieeexplore.ieee.org
Deep Neural Networks (DNNs) have been widely used in various artificial intelligence (AI)
applications due to their overwhelming performance. Furthermore, recently, several …

An Energy-Efficient Deep Convolutional Neural Network Training Accelerator for In Situ Personalization on Smart Devices

S Choi, J Sim, M Kang, Y Choi, H Kim… - IEEE Journal of Solid …, 2020 - ieeexplore.ieee.org
A scalable deep-learning accelerator supporting the training process is implemented for
device personalization of deep convolutional neural networks (CNNs). It consists of three …

Neuron activation coverage: Rethinking out-of-distribution detection and generalization

Y Liu, CX Tian, H Li, L Ma, S Wang - arXiv preprint arXiv:2306.02879, 2023 - arxiv.org
The out-of-distribution (OOD) problem generally arises when neural networks encounter
data that significantly deviates from the training data distribution, ie, in-distribution (InD). In …

THETA: A high-efficiency training accelerator for DNNs with triple-side sparsity exploration

J Lu, J Huang, Z Wang - … on Very Large Scale Integration (VLSI …, 2022 - ieeexplore.ieee.org
Training deep neural networks (DNNs) on edge devices has attracted increasing attention in
real-world applications for domain adaption and privacy protection. However, deploying …

Efficient-grad: Efficient training deep convolutional neural networks on edge devices with grad ient optimizations

Z Hong, CP Yue - ACM Transactions on Embedded Computing Systems …, 2022 - dl.acm.org
With the prospering of mobile devices, the distributed learning approach, enabling model
training with decentralized data, has attracted great interest from researchers. However, the …

Prediction confidence based low complexity gradient computation for accelerating DNN training

D Shin, G Kim, J Jo, J Park - 2020 57th ACM/IEEE Design …, 2020 - ieeexplore.ieee.org
In deep neural network (DNN) training, network weights are iteratively updated with the
weight gradients that are obtained from stochastic gradient descent (SGD). Since SGD …

Low complexity gradient computation techniques to accelerate deep neural network training

D Shin, G Kim, J Jo, J Park - IEEE Transactions on Neural …, 2021 - ieeexplore.ieee.org
Deep neural network (DNN) training is an iterative process of updating network weights,
called gradient computation, where (mini-batch) stochastic gradient descent (SGD) …

A deep neural network training architecture with inference-aware heterogeneous data-type

S Choi, J Shin, LS Kim - IEEE Transactions on Computers, 2021 - ieeexplore.ieee.org
As deep learning applications often encounter accuracy degradation due to the distorted
inputs from a variety of environmental conditions, training with personal data has become …

Nebula: A Scalable and Flexible Accelerator for DNN Multi-Branch Blocks on Embedded Systems

D Yang, X Li, L Qi, W Zhang, Z Jiang - Electronics, 2022 - mdpi.com
Deep neural networks (DNNs) are widely used in many artificial intelligence applications;
many specialized DNN-inference accelerators have been proposed. However, existing DNN …

Exploiting activation based gradient output sparsity to accelerate backpropagation in cnns

A Sarma, S Singh, H Jiang, A Pattnaik… - arXiv preprint arXiv …, 2021 - arxiv.org
Machine/deep-learning (ML/DL) based techniques are emerging as a driving force behind
many cutting-edge technologies, achieving high accuracy on computer vision workloads …