Prevalence of neural collapse during the terminal phase of deep learning training

V Papyan, XY Han, DL Donoho - Proceedings of the …, 2020 - National Acad Sciences
Modern practice for training classification deepnets involves a terminal phase of training
(TPT), which begins at the epoch where training error first vanishes. During TPT, the training …

Algorithm unrolling: Interpretable, efficient deep learning for signal and image processing

V Monga, Y Li, YC Eldar - IEEE Signal Processing Magazine, 2021 - ieeexplore.ieee.org
Deep neural networks provide unprecedented performance gains in many real-world
problems in signal and image processing. Despite these gains, the future development and …

Learning to optimize: A primer and a benchmark

T Chen, X Chen, W Chen, H Heaton, J Liu… - Journal of Machine …, 2022 - jmlr.org
Learning to optimize (L2O) is an emerging approach that leverages machine learning to
develop optimization methods, aiming at reducing the laborious iterations of hand …

The difficulty of computing stable and accurate neural networks: On the barriers of deep learning and Smale's 18th problem

MJ Colbrook, V Antun… - Proceedings of the …, 2022 - National Acad Sciences
Deep learning (DL) has had unprecedented success and is now entering scientific
computing with full force. However, current DL methods typically suffer from instability, even …

Model-driven deep unrolling: Towards interpretable deep learning against noise attacks for intelligent fault diagnosis

Z Zhao, T Li, B An, S Wang, B Ding, R Yan, X Chen - ISA transactions, 2022 - Elsevier
Intelligent fault diagnosis (IFD) has experienced tremendous progress owing to a great deal
to deep learning (DL)-based methods over the decades. However, the “black box” nature of …

Interpretable neural network via algorithm unrolling for mechanical fault diagnosis

B An, S Wang, Z Zhao, F Qin, R Yan… - IEEE Transactions on …, 2022 - ieeexplore.ieee.org
Artificial neural network (ANN) has achieved great success in mechanical fault diagnosis
and has been widely used. However, traditional ANN is still opaque in terms of …

[HTML][HTML] Regularization by architecture: A deep prior approach for inverse problems

S Dittmer, T Kluth, P Maass, D Otero Baguer - Journal of Mathematical …, 2020 - Springer
The present paper studies so-called deep image prior (DIP) techniques in the context of ill-
posed inverse problems. DIP networks have been recently introduced for applications in …

Learning the sparse prior: Modern approaches

GJ Peng - Wiley Interdisciplinary Reviews: Computational …, 2024 - Wiley Online Library
The sparse prior has been widely adopted to establish data models for numerous
applications. In this context, most of them are based on one of three foundational paradigms …

Adversarial algorithm unrolling network for interpretable mechanical anomaly detection

B An, S Wang, F Qin, Z Zhao, R Yan… - IEEE Transactions on …, 2023 - ieeexplore.ieee.org
In mechanical anomaly detection, algorithms with higher accuracy, such as those based on
artificial neural networks, are frequently constructed as black boxes, resulting in opaque …

Deep scattering network with fractional wavelet transform

J Shi, Y Zhao, W Xiang, V Monga… - IEEE Transactions on …, 2021 - ieeexplore.ieee.org
Deep convolutional neural networks (DCNNs) have recently emerged as a powerful tool to
deliver breakthrough performances in various image analysis and processing applications …