Amazon sagemaker model monitor: A system for real-time insights into deployed machine learning models

D Nigenda, Z Karnin, MB Zafar, R Ramesha… - Proceedings of the 28th …, 2022 - dl.acm.org
With the increasing adoption of machine learning (ML) models and systems in high-stakes
settings across different industries, guaranteeing a model's performance after deployment …

Model monitoring in practice: Lessons learned and open challenges

K Kenthapadi, H Lakkaraju, P Natarajan… - Proceedings of the 28th …, 2022 - dl.acm.org
Artificial Intelligence (AI) is increasingly playing an integral role in determining our day-to-
day experiences. Increasingly, the applications of AI are no longer limited to search and …

Bounded space differentially private quantiles

D Alabi, O Ben-Eliezer, A Chaturvedi - arXiv preprint arXiv:2201.03380, 2022 - arxiv.org
Estimating the quantiles of a large dataset is a fundamental problem in both the streaming
algorithms literature and the differential privacy literature. However, all existing private …

A human-centric perspective on model monitoring

MN Shergadwala, H Lakkaraju… - Proceedings of the AAAI …, 2022 - ojs.aaai.org
Predictive models are increasingly used to make various consequential decisions in high-
stakes domains such as healthcare, finance, and policy. It becomes critical to ensure that …

Optimizing Data Pipelines for Machine Learning in Feature Stores

R Liu, K Park, F Psallidas, X Zhu, J Mo, R Sen… - Proceedings of the …, 2023 - dl.acm.org
Data pipelines (ie, converting raw data to features) are critical for machine learning (ML)
models, yet their development and management is time-consuming. Feature stores have …

CORE-Sketch: On Exact Computation of Median Absolute Deviation with Limited Space

H Guan, Z Chen, S Song - Proceedings of the VLDB Endowment, 2023 - dl.acm.org
Median absolute deviation (MAD), the median of the absolute deviations from the median,
has been found useful in various applications such as outlier detection. Together with …

Together is better: Heavy hitters quantile estimation

R Shahout, R Friedman, R Ben Basat - … of the ACM on Management of …, 2023 - dl.acm.org
Stream monitoring is fundamental in many data stream applications, such as financial data
trackers, security, anomaly detection, and load balancing. In that respect, quantiles are of …

Applications of sketching and pathways to impact

G Cormode - Proceedings of the 42nd ACM SIGMOD-SIGACT-SIGAI …, 2023 - dl.acm.org
Data summaries (aka, sketches) are compact data structures that can be updated flexibly
and efficiently to capture certain properties of a data set. Well-known examples include set …

Current trends in data summaries

G Cormode - ACM SIGMOD Record, 2022 - dl.acm.org
The research area of data summarization seeks to find small data structures that can be
updated flexibly, and answer certain queries on the input accurately. Summaries are widely …

A Human-Centric Take on Model Monitoring

MN Shergadwala, H Lakkaraju… - arXiv preprint arXiv …, 2022 - arxiv.org
Predictive models are increasingly used to make various consequential decisions in high-
stakes domains such as healthcare, finance, and policy. It becomes critical to ensure that …