Spatten: Efficient sparse attention architecture with cascade token and head pruning

H Wang, Z Zhang, S Han - 2021 IEEE International Symposium …, 2021 - ieeexplore.ieee.org
The attention mechanism is becoming increasingly popular in Natural Language Processing
(NLP) applications, showing superior performance than convolutional and recurrent …

A survey of accelerating parallel sparse linear algebra

G Xiao, C Yin, T Zhou, X Li, Y Chen, K Li - ACM Computing Surveys, 2023 - dl.acm.org
Sparse linear algebra includes the fundamental and important operations in various large-
scale scientific computing and real-world applications. There exists performance bottleneck …

Optimization of Large-Scale Sparse Matrix-Vector Multiplication on Multi-GPU Systems

J Gao, W Ji, Y Wang - ACM Transactions on Architecture and Code …, 2024 - dl.acm.org
Sparse matrix-vector multiplication (SpMV) is one of the important kernels of many iterative
algorithms for solving sparse linear systems. The limited storage and computational …

Albus: A method for efficiently processing spmv using simd and load balancing

H Bian, J Huang, L Liu, D Huang, X Wang - Future Generation Computer …, 2021 - Elsevier
SpMV (Sparse matrix–vector multiplication) is widely used in many fields. Improving the
performance of SpMV has been the pursuit of many researchers. Parallel SpMV using multi …

SOFA: A compute-memory optimized sparsity accelerator via cross-stage coordinated tiling

H Wang, J Fang, X Tang, Z Yue, J Li… - 2024 57th IEEE/ACM …, 2024 - ieeexplore.ieee.org
Benefiting from the self-attention mechanism, Transformer models have attained impressive
contextual comprehension capabilities for lengthy texts. The requirements of high …

Efficient Algorithm Design of Optimizing SpMV on GPU

G Chu, Y He, L Dong, Z Ding, D Chen, H Bai… - Proceedings of the …, 2023 - dl.acm.org
Sparse matrix-vector multiplication (SpMV) is a fundamental building block for various
numerical computing applications. However, most existing GPU-SpMV approaches may …

A Systematic Literature Survey of Sparse Matrix-Vector Multiplication

J Gao, B Liu, W Ji, H Huang - arXiv preprint arXiv:2404.06047, 2024 - arxiv.org
Sparse matrix-vector multiplication (SpMV) is a crucial computing kernel with widespread
applications in iterative algorithms. Over the past decades, research on SpMV optimization …

Performance enhancement strategies for sparse matrix-vector multiplication (spmv) and iterative linear solvers

T Mohammed, R Mehmood - arXiv preprint arXiv:2212.07490, 2022 - arxiv.org
Iterative solutions of sparse linear systems and sparse eigenvalue problems have a
fundamental role in vital fields of scientific research and engineering. The crucial computing …

Implementation and optimization of SpMV algorithm based on SW26010P many-core processor and stored in BCSR format

M Ma, X Huang, J Xu, D Jia - Scientific Reports, 2024 - nature.com
The irregular distribution of non-zero elements of large-scale sparse matrix leads to low data
access efficiency caused by the unique architecture of the Sunway many-core processor …

Lightweight Deep Learning for Missing Data Imputation in Wastewater Treatment With Variational Residual Auto-Encoder

W Zhang, R Li, P Quan, J Chang… - IEEE Internet of Things …, 2024 - ieeexplore.ieee.org
Deep variational residual auto-encoder (ResNet-VAE) has shown promising outcomes in
missing imputation of wastewater quality data. Nevertheless, with its large storage size and …