Decision-Focused Uncertainty Quantification

S Cortes-Gomez, C Patiño, Y Byun, S Wu… - arXiv preprint arXiv …, 2024 - arxiv.org
arXiv preprint arXiv:2410.01767, 2024arxiv.org
There is increasing interest in''decision-focused''machine learning methods which train
models to account for how their predictions are used in downstream optimization problems.
Doing so can often improve performance on subsequent decision problems. However,
current methods for uncertainty quantification do not incorporate any information at all about
downstream decisions. We develop a framework based on conformal prediction to produce
prediction sets that account for a downstream decision loss function, making them more …
There is increasing interest in ''decision-focused'' machine learning methods which train models to account for how their predictions are used in downstream optimization problems. Doing so can often improve performance on subsequent decision problems. However, current methods for uncertainty quantification do not incorporate any information at all about downstream decisions. We develop a framework based on conformal prediction to produce prediction sets that account for a downstream decision loss function, making them more appropriate to inform high-stakes decision-making. Our approach harnesses the strengths of conformal methods--modularity, model-agnosticism, and statistical coverage guarantees--while incorporating downstream decisions and user-specified utility functions. We prove that our methods retain standard coverage guarantees. Empirical evaluation across a range of datasets and utility metrics demonstrates that our methods achieve significantly lower decision loss compared to standard conformal methods. Additionally, we present a real-world use case in healthcare diagnosis, where our method effectively incorporates the hierarchical structure of dermatological diseases. It successfully generates sets with coherent diagnostic meaning, aiding the triage process during dermatology diagnosis and illustrating how our method can ground high-stakes decision-making on external domain knowledge.
arxiv.org
以上显示的是最相近的搜索结果。 查看全部搜索结果

Google学术搜索按钮

example.edu/paper.pdf
搜索
获取 PDF 文件
引用
References