作者
Andrew D Ferguson, Peter Bodik, Srikanth Kandula, Eric Boutin, Rodrigo Fonseca
发表日期
2012
研讨会论文
EuroSys
简介
Data processing frameworks such as MapReduce [8] and Dryad [11] are used today in business environments where customers expect guaranteed performance. To date, however, these systems are not capable of providing guarantees on job latency because scheduling policies are based on fair-sharing, and operators seek high cluster use through statistical multiplexing and over-subscription. With Jockey, we provide latency SLOs for data parallel jobs written in SCOPE. Jockey precomputes statistics using a simulator that captures the job's complex internal dependencies, accurately and efficiently predicting the remaining run time at different resource allocations and in different stages of the job. Our control policy monitors a job's performance, and dynamically adjusts resource allocation in the shared cluster in order to maximize the job's economic utility while minimizing its impact on the rest of the cluster. In our …
学术搜索中的文章
AD Ferguson, P Bodik, S Kandula, E Boutin, R Fonseca - Proceedings of the 7th ACM european conference on …, 2012