BIGhybrid: a simulator for MapReduce applications in hybrid distributed infrastructures validated with the Grid5000 experimental platform

JCS Anjos, G Fedak, CFR Geyer - … and Computation: Practice …, 2016 - Wiley Online Library
Concurrency and Computation: Practice and Experience, 2016Wiley Online Library
Cloud computing has increasingly been used as a platform for running large business and
data processing applications. Conversely, Desktop Grids have been successfully employed
in a wide range of projects, because they are able to take advantage of a large number of
resources provided free of charge by volunteers. A hybrid infrastructure created from the
combination of Cloud and Desktop Grids infrastructures can provide a low‐cost and scalable
solution for Big Data analysis. Although frameworks like MapReduce have been designed to …
Summary
Cloud computing has increasingly been used as a platform for running large business and data processing applications. Conversely, Desktop Grids have been successfully employed in a wide range of projects, because they are able to take advantage of a large number of resources provided free of charge by volunteers. A hybrid infrastructure created from the combination of Cloud and Desktop Grids infrastructures can provide a low‐cost and scalable solution for Big Data analysis. Although frameworks like MapReduce have been designed to exploit commodity hardware, their ability to take advantage of a hybrid infrastructure poses significant challenges because of their large resource heterogeneity and high churn rate. In this paper, BIGhybrid is proposed, a simulator for two existing classes of MapReduce runtime environments: BitDew‐MapReduce designed for Desktop Grids and BlobSeer‐Hadoop designed for Cloud computing, where the goal is to carry out accurate simulations of MapReduce executions in a hybrid infrastructure composed of Cloud computing and Desktop Grid resources. This work describes the principles of the simulator and describes the validation of BIGhybrid with the Grid5000 experimental platform. Owing to BIGhybrid, developers can investigate and evaluate new algorithms to enable MapReduce to be executed in hybrid infrastructures. This includes topics such as resource allocation and data splitting. Copyright © 2015 John Wiley & Sons, Ltd.
Wiley Online Library
以上显示的是最相近的搜索结果。 查看全部搜索结果