Flashattention: Fast and memory-efficient exact attention with io-awareness

T Dao, D Fu, S Ermon, A Rudra… - Advances in Neural …, 2022 - proceedings.neurips.cc
Transformers are slow and memory-hungry on long sequences, since the time and memory
complexity of self-attention are quadratic in sequence length. Approximate attention …

Synopses for massive data: Samples, histograms, wavelets, sketches

G Cormode, M Garofalakis, PJ Haas… - … and Trends® in …, 2011 - nowpublishers.com
Abstract Methods for Approximate Query Processing (AQP) are essential for dealing with
massive data. They are often the only means of providing interactive response times when …

An improved data stream summary: the count-min sketch and its applications

G Cormode, S Muthukrishnan - Journal of Algorithms, 2005 - Elsevier
We introduce a new sublinear space data structure—the count-min sketch—for summarizing
data streams. Our sketch allows fundamental queries in data stream summarization such as …

Data streams: Algorithms and applications

S Muthukrishnan - Foundations and Trends® in Theoretical …, 2005 - nowpublishers.com
In the data stream scenario, input arrives very rapidly and there is limited memory to store
the input. Algorithms have to work with one or few passes over the data, space less than …

Sketching and sublinear data structures in genomics

G Marçais, B Solomon, R Patro… - Annual Review of …, 2019 - annualreviews.org
Large-scale genomics demands computational methods that scale sublinearly with the
growth of data. We review several data structures and sketching techniques that have been …

An optimal algorithm for the distinct elements problem

DM Kane, J Nelson, DP Woodruff - Proceedings of the twenty-ninth ACM …, 2010 - dl.acm.org
We give the first optimal algorithm for estimating the number of distinct elements in a data
stream, closing a long line of theoretical research on this problem begun by Flajolet and …

A framework for adversarially robust streaming algorithms

O Ben-Eliezer, R Jayaram, DP Woodruff… - ACM Journal of the ACM …, 2022 - dl.acm.org
We investigate the adversarial robustness of streaming algorithms. In this context, an
algorithm is considered robust if its performance guarantees hold even if the stream is …

An improved data stream summary: The count-min sketch and its applications

G Cormode, S Muthukrishnan - … , Buenos Aires, Argentina, April 5-8, 2004 …, 2004 - Springer
We introduce a new sublinear space data structure—the Count-Min Sketch—for
summarizing data streams. Our sketch allows fundamental queries in data stream …

Tight bounds for adversarially robust streams and sliding windows via difference estimators

DP Woodruff, S Zhou - 2021 IEEE 62nd Annual Symposium on …, 2022 - ieeexplore.ieee.org
In the adversarially robust streaming model, a stream of elements is presented to an
algorithm and is allowed to depend on the output of the algorithm at earlier times during the …

An optimal lower bound on the communication complexity of gap-hamming-distance

A Chakrabarti, O Regev - Proceedings of the forty-third annual ACM …, 2011 - dl.acm.org
We prove an optimal Ω (n) lower bound on the randomized communication complexity of the
much-studied Gap-Hamming-Distance problem. As a consequence, we obtain essentially …