Jobflow: Computational workflows made simple

AS Rosen, M Gallant, J George, J Riebesell… - Journal of Open …, 2024 - joss.theoj.org
We present Jobflow, a domain-agnostic Python package for writing computational workflows
tailored for high-throughput computing applications. With its simple decorator-based …

PSI/J: A portable interface for submitting, monitoring, and managing jobs

M Hategan-Marandiuc, A Merzky… - 2023 IEEE 19th …, 2023 - ieeexplore.ieee.org
It is generally desirable for high-performance computing (HPC) applications to be portable
between HPC systems, for example to make use of more performant hardware, make …

ProvLight: Efficient workflow provenance capture on the edge-to-cloud continuum

D Rosendo, M Mattoso, A Costan… - 2023 IEEE …, 2023 - ieeexplore.ieee.org
Modern scientific workflows require hybrid infrastructures combining numerous
decentralized resources on the IoT/Edge interconnected to Cloud/HPC systems (aka the …

Automated generation of scientific workflow generators with WfChef

T Coleman, H Casanova, RF Da Silva - Future Generation Computer …, 2023 - Elsevier
Scientific workflow applications have gained significant importance, and their automated
and efficient execution on large-scale computing platforms has been the subject of extensive …

Driving Next-Generation Workflows from the Data Plane

F Suter, RF Da Silva, A Gainaru… - 2023 IEEE 19th …, 2023 - ieeexplore.ieee.org
We observe the emergence of a new generation of scientific workflows that process data
produced at a sustained rate by scientific instruments and large scale numerical simulations …

UniFaaS: Programming across Distributed Cyberinfrastructure with Federated Function Serving

Y Li, R Chard, Y Babuji, K Chard, I Foster… - arXiv preprint arXiv …, 2024 - arxiv.org
Modern scientific applications are increasingly decomposable into individual functions that
may be deployed across distributed and diverse cyberinfrastructure such as …

WfCommons: Data Collection and Runtime Experiments using Multiple Workflow Systems

H Casanova, K Berney, S Chastel… - 2023 IEEE 47th …, 2023 - ieeexplore.ieee.org
Scientific workflows have become ubiquitous across scientific fields, and their execution
methods and systems continue to be the subject of research and development. Most …

Towards Lightweight Data Integration Using Multi-Workflow Provenance and Data Observability

R Souza, TJ Skluzacek, SR Wilkinson… - 2023 IEEE 19th …, 2023 - ieeexplore.ieee.org
Modern large-scale scientific discovery requires multidisciplinary collaboration across
diverse computing facilities, including High Performance Computing (HPC) machines and …

Julia as a unifying end-to-end workflow language on the Frontier exascale system

WF Godoy, P Valero-Lara, C Anderson… - Proceedings of the SC' …, 2023 - dl.acm.org
We evaluate Julia as a single language and ecosystem paradigm powered by LLVM to
develop workflow components for high-performance computing. We run a Gray-Scott, 2 …

Predicting Dynamic Memory Requirements for Scientific Workflow Tasks

J Bader, N Diedrich, L Thamsen… - 2023 IEEE International …, 2023 - ieeexplore.ieee.org
With the increasing amount of data available to scientists in disciplines as diverse as
bioinformatics, physics, and remote sensing, scientific workflow systems are becoming …