[PDF][PDF] Training nonlinear transformers for efficient in-context learning: A theoretical learning and generalization analysis
Transformer-based large language models have displayed impressive in-context learning
capabilities, where a pre-trained model can handle new tasks without fine-tuning by simply …
capabilities, where a pre-trained model can handle new tasks without fine-tuning by simply …
Learning on transformers is provable low-rank and sparse: A one-layer analysis
Efficient training and inference algorithms, such as low-rank adaption and model pruning,
have shown impressive performance for learning Transformer-based large foundation …
have shown impressive performance for learning Transformer-based large foundation …
Quantifying lottery tickets under label noise: accuracy, calibration, and complexity
Pruning deep neural networks is a widely used strategy to alleviate the computational
burden in machine learning. Overwhelming empirical evidence suggests that pruned …
burden in machine learning. Overwhelming empirical evidence suggests that pruned …
How Do Nonlinear Transformers Learn and Generalize in In-Context Learning?
Transformer-based large language models have displayed impressive in-context learning
capabilities, where a pre-trained model can handle new tasks without fine-tuning by simply …
capabilities, where a pre-trained model can handle new tasks without fine-tuning by simply …
How Sparse Can We Prune A Deep Network: A Fundamental Limit Viewpoint
Q Zhang, R Zhang, J Sun, Y Liu - arXiv preprint arXiv:2306.05857, 2023 - arxiv.org
Network pruning is an effective measure to alleviate the storage and computational burden
of deep neural networks arising from its high overparameterization. Thus raises a …
of deep neural networks arising from its high overparameterization. Thus raises a …
How Sparse Can We Prune A Deep Network: A Geometric Viewpoint
Q Zhang, R Zhang, J Sun, Y Liu - 2023 - openreview.net
Network pruning constitutes an effective measure to alleviate the storage and computational
burden of deep neural networks which arises from its overparameterization. A fundamental …
burden of deep neural networks which arises from its overparameterization. A fundamental …