Evaluating scientific workflow engines for data and compute intensive discoveries

R Singh, JA Graves, V Anantharaj… - … Conference on Big …, 2019 - ieeexplore.ieee.org
R Singh, JA Graves, V Anantharaj, SR Sukumar
2019 IEEE International Conference on Big Data (Big Data), 2019ieeexplore.ieee.org
Workflow engines used to script scientific experiments involving numerical simulation, data
analysis, instruments, edge sensors, and artificial intelligence have to deal with the
complexities of hardware, software, resource availability, and the collaborative nature of
science. In this paper, we survey workflow engines used in data-intensive and compute-
intensive discovery pipelines from scientific disciplines such as astronomy, high energy
physics, earth system science, bio-medicine, and material science and present a qualitative …
Workflow engines used to script scientific experiments involving numerical simulation, data analysis, instruments, edge sensors, and artificial intelligence have to deal with the complexities of hardware, software, resource availability, and the collaborative nature of science. In this paper, we survey workflow engines used in data-intensive and compute-intensive discovery pipelines from scientific disciplines such as astronomy, high energy physics, earth system science, bio-medicine, and material science and present a qualitative analysis of their respective capabilities. We compare 5 popular workflow engines and their differentiated approach to job orchestration, job launching, data management and provenance, security authentication, ease-of-use, workflow description, and scripting semantics. The comparisons presented in this paper allow practitioners to choose the appropriate engine for their scientific experiment and lead to recommendations for future work.
ieeexplore.ieee.org
以上显示的是最相近的搜索结果。 查看全部搜索结果

Google学术搜索按钮

example.edu/paper.pdf
搜索
获取 PDF 文件
引用
References