Online optimization of file transfers in high-speed networks

M Arifuzzaman, E Arslan - … of the International Conference for High …, 2021 - dl.acm.org
File transfers in high-speed networks require network and I/O parallelism to reach high
speeds, however, creating arbitrarily large numbers of I/O and network threads overwhelms …

A view from ORNL: Scientific data research opportunities in the big data age

S Klasky, M Wolf, M Ainsworth, C Atkins… - 2018 IEEE 38th …, 2018 - ieeexplore.ieee.org
One of the core issues across computer and computational science today is adapting to,
managing, and learning from the influx of" Big Data". In the commercial space, this problem …

Canopus: A paradigm shift towards elastic extreme-scale data analytics on hpc storage

T Lu, E Suchyta, D Pugmire, J Choi… - 2017 IEEE …, 2017 - ieeexplore.ieee.org
Scientific simulations on high performance computing (HPC) platforms generate large
quantities of data. To bridge the widening gap between compute and I/O, and enable data to …

Staging based task execution for data-driven, in-situ scientific workflows

Z Wang, P Subedi, M Dorier, PE Davis… - … on Cluster Computing …, 2020 - ieeexplore.ieee.org
As scientific workflows increasingly use extreme-scale resources, the imbalance between
higher computational capabilities, generated data volumes, and available I/O bandwidth is …

[PDF][PDF] In situ visualization techniques for high performance computing

J Kress - Eugene OR, 2017 - cs.uoregon.edu
Scientific visualization for exascale computing is very likely to require in situ processing.
Traditional simulation checkpointing and post hoc visualization will likely be unsustainable …

Exacution: Enhancing scientific data management for exascale

S Klasky, E Suchyta, M Ainsworth, Q Liu… - 2017 IEEE 37th …, 2017 - ieeexplore.ieee.org
As we continue toward exascale, scientific data volume is continuing to scale and becoming
more burdensome to manage. In this paper, we lay out opportunities to enhance state of the …

Binning based data reduction for vector field data of a particle-in-cell fusion simulation

J Kress, J Choi, S Klasky, M Churchill, H Childs… - … Computing: ISC High …, 2018 - Springer
With this work, we explore the feasibility of using in situ data binning techniques to achieve
significant data reductions for particle data, and study the associated errors for several post …

In-line vs. in-transit in situ: Which technique to use at scale?

J Kress - 2020 - scholarsbank.uoregon.edu
In situ visualization is increasingly necessary to address I/O limitations on supercomputers.
With the increasing heterogeneity of supercomputer design, efficient and cost effective use …

Facilitating staging-based unstructured mesh processing to support hybrid in-situ workflows

Z Wang, P Subedi, M Dorier, PE Davis… - 2021 IEEE …, 2021 - ieeexplore.ieee.org
In-situ and in-transit processing alleviate the gap between the computing and I/O capabilities
by scheduling data analytics close to the data source. Hybrid in-situ processing splits data …

Sirius: Enabling progressive data exploration for extreme-scale scientific data

Z Qiao, T Lu, H Luo, Q Liu, S Klasky… - … on Multi-Scale …, 2018 - ieeexplore.ieee.org
Scientific simulations on high performance computing (HPC) platforms generate large
quantities of data. To bridge the widening gap between compute and I/O, and enable data to …