Data pruning and neural scaling laws: fundamental limitations of score-based algorithms

F Ayed, S Hayou - arXiv preprint arXiv:2302.06960, 2023 - arxiv.org
Data pruning algorithms are commonly used to reduce the memory and computational cost
of the optimization process. Recent empirical results reveal that random data pruning …