Olive: Accelerating large language models via hardware-friendly outlier-victim pair quantization
Transformer-based large language models (LLMs) have achieved great success with the
growing model size. LLMs' size grows by 240× every two years, which outpaces the …
growing model size. LLMs' size grows by 240× every two years, which outpaces the …
Rptq: Reorder-based post-training quantization for large language models
Large-scale language models (LLMs) have demonstrated impressive performance, but their
deployment presents challenges due to their significant memory usage. This issue can be …
deployment presents challenges due to their significant memory usage. This issue can be …
GMLake: Efficient and Transparent GPU Memory Defragmentation for Large-scale DNN Training with Virtual Memory Stitching
Large-scale deep neural networks (DNNs), such as large language models (LLMs), have
revolutionized the artificial intelligence (AI) field and become increasingly popular. However …
revolutionized the artificial intelligence (AI) field and become increasingly popular. However …
JUNO: Optimizing High-Dimensional Approximate Nearest Neighbour Search with Sparsity-Aware Algorithm and Ray-Tracing Core Mapping
Approximate nearest neighbor (ANN) search is a widely applied technique in modern
intelligent applications, such as recommendation systems and vector databases. Therefore …
intelligent applications, such as recommendation systems and vector databases. Therefore …
MSD: Mixing Signed Digit Representations for Hardware-efficient DNN Acceleration on FPGA with Heterogeneous Resources
By quantizing weights with different precision for different parts of a network, mixed-precision
quantization promises to reduce the hardware cost and improve the speed of deep neural …
quantization promises to reduce the hardware cost and improve the speed of deep neural …
Dybit: Dynamic bit-precision numbers for efficient quantized neural network inference
To accelerate the inference of deep neural networks (DNNs), quantization with low-bitwidth
numbers is actively researched. A prominent challenge is to quantize the DNN models into …
numbers is actively researched. A prominent challenge is to quantize the DNN models into …
Approximate computing survey, Part II: Application-specific & architectural approximation techniques and applications
V Leon, MA Hanif, G Armeniakos, X Jiao… - arXiv preprint arXiv …, 2023 - arxiv.org
The challenging deployment of compute-intensive applications from domains such Artificial
Intelligence (AI) and Digital Signal Processing (DSP), forces the community of computing …
Intelligence (AI) and Digital Signal Processing (DSP), forces the community of computing …
Lut-nn: Empower efficient neural network inference with centroid learning and table lookup
On-device Deep Neural Network (DNN) inference consumes significant computing
resources and development efforts. To alleviate that, we propose LUT-NN, the first system to …
resources and development efforts. To alleviate that, we propose LUT-NN, the first system to …
Nesting forward automatic differentiation for memory-efficient deep neural network training
An activation function is an element-wise mathematical function and plays a crucial role in
deep neural networks (DNN). Many novel and sophisticated activation functions have been …
deep neural networks (DNN). Many novel and sophisticated activation functions have been …
Accelerating sparse dnns based on tiled gemm
Network pruning can reduce the computation cost of deep neural network (DNN) models.
However, sparse models often produce randomly-distributed weights to maintain accuracy …
However, sparse models often produce randomly-distributed weights to maintain accuracy …