Vision transformers (ViTs) have recently received explosive popularity, but their enormous model sizes and training costs remain daunting. Conventional post-training pruning often …
With graphs rapidly growing in size and deeper graph neural networks (GNNs) emerging, the training and inference of GNNs become increasingly expensive. Existing network weight …
The computer vision world has been re-gaining enthusiasm in various pre-trained models, including both classical ImageNet supervised pre-training and recently emerged self …
The deployment constraints in practical applications necessitate the pruning of large-scale deep learning models, ie, promoting their weight sparsity. As illustrated by the Lottery Ticket …
There have been long-standing controversies and inconsistencies over the experiment setup and criteria for identifying the" winning ticket" in literature. To reconcile such, we revisit …
The lottery ticket hypothesis (LTH) has shown that dense models contain highly sparse subnetworks (ie, winning tickets) that can be trained in isolation to match full accuracy …
Z Zhang, X Chen, T Chen… - … Conference on Machine …, 2021 - proceedings.mlr.press
The lottery ticket hypothesis (LTH) reveals the existence of winning tickets (sparse but critical subnetworks) for dense networks, that can be trained in isolation from random initialization …
Trojan attacks threaten deep neural networks (DNNs) by poisoning them to behave normally on most samples, yet to produce manipulated results for inputs attached with a particular …
Large-scale pre-training has recently revolutionized vision-and-language (VL) research. Models such as LXMERT and UNITER have significantly lifted the state of the art over a wide …