A comprehensive survey of dataset distillation

S Lei, D Tao - IEEE Transactions on Pattern Analysis and …, 2023 - ieeexplore.ieee.org
Deep learning technology has developed unprecedentedly in the last decade and has
become the primary choice in many application domains. This progress is mainly attributed …

You only condense once: Two rules for pruning condensed datasets

Y He, L Xiao, JT Zhou - Advances in Neural Information …, 2023 - proceedings.neurips.cc
Dataset condensation is a crucial tool for enhancing training efficiency by reducing the size
of the training dataset, particularly in on-device scenarios. However, these scenarios have …

Towards lossless dataset distillation via difficulty-aligned trajectory matching

Z Guo, K Wang, G Cazenavette, H Li, K Zhang… - arXiv preprint arXiv …, 2023 - arxiv.org
The ultimate goal of Dataset Distillation is to synthesize a small synthetic dataset such that a
model trained on this synthetic set will perform equally well as a model trained on the full …

Sequential subset matching for dataset distillation

J Du, Q Shi, JT Zhou - Advances in Neural Information …, 2024 - proceedings.neurips.cc
Dataset distillation is a newly emerging task that synthesizes a small-size dataset used in
training deep neural networks (DNNs) for reducing data storage and model training costs …

Ideal: Influence-driven selective annotations empower in-context learners in large language models

S Zhang, X Xia, Z Wang, LH Chen, J Liu, Q Wu… - arXiv preprint arXiv …, 2023 - arxiv.org
In-context learning is a promising paradigm that utilizes in-context examples as prompts for
the predictions of large language models. These prompts are crucial for achieving strong …

Mgdd: A meta generator for fast dataset distillation

S Liu, X Wang - Advances in Neural Information Processing …, 2024 - proceedings.neurips.cc
Existing dataset distillation (DD) techniques typically rely on iterative strategies to synthesize
condensed datasets, where datasets before and after distillation are forward and backward …

Efficient dataset distillation via minimax diffusion

J Gu, S Vahidian, V Kungurtsev… - Proceedings of the …, 2024 - openaccess.thecvf.com
Dataset distillation reduces the storage and computational consumption of training a
network by generating a small surrogate dataset that encapsulates rich information of the …

Multimodal dataset distillation for image-text retrieval

X Wu, Z Deng, O Russakovsky - arXiv preprint arXiv:2308.07545, 2023 - arxiv.org
Dataset distillation methods offer the promise of reducing a large-scale dataset down to a
significantly smaller set of (potentially synthetic) training examples, which preserve sufficient …

Graph data condensation via self-expressive graph structure reconstruction

Z Liu, C Zeng, G Zheng - Proceedings of the 30th ACM SIGKDD …, 2024 - dl.acm.org
With the increasing demands of training graph neural networks (GNNs) on large-scale
graphs, graph data condensation has emerged as a critical technique to relieve the storage …

Few-shot dataset distillation via translative pre-training

S Liu, X Wang - Proceedings of the IEEE/CVF International …, 2023 - openaccess.thecvf.com
Dataset distillation aims at a small synthetic dataset to mimic the training performance on
neural networks of a given large dataset. Existing approaches heavily rely on an iterative …