Remote sensing big data computing: Challenges and opportunities

Y Ma, H Wu, L Wang, B Huang, R Ranjan… - Future Generation …, 2015 - Elsevier
As we have entered an era of high resolution earth observation, the RS data are undergoing
an explosive growth. The proliferation of data also give rise to the increasing complexity of …

Recent developments in parallel and distributed computing for remotely sensed big data processing

Z Wu, J Sun, Y Zhang, Z Wei… - Proceedings of the …, 2021 - ieeexplore.ieee.org
This article gives a survey of state-of-the-art methods for processing remotely sensed big
data and thoroughly investigates existing parallel implementations on diverse popular high …

Ceph: A scalable, high-performance distributed file system

S Weil, SA Brandt, EL Miller, DDE Long… - Proceedings of the 7th …, 2006 - usenix.org
We have developed Ceph, a distributed file system that provides excellent performance,
reliability, and scalability. Ceph maximizes the separation between data and metadata …

Data sieving and collective I/O in ROMIO

R Thakur, W Gropp, E Lusk - Proceedings. Frontiers' 99 …, 1999 - ieeexplore.ieee.org
The I/O access patterns of parallel programs often consist of accesses to a large number of
small, noncontiguous pieces of data. If an application's I/O needs are met by making many …

Understanding and improving computational science storage access through continuous characterization

P Carns, K Harms, W Allcock, C Bacon, S Lang… - ACM Transactions on …, 2011 - dl.acm.org
Computational science applications are driving a demand for increasingly powerful storage
systems. While many techniques are available for capturing the I/O behavior of individual …

On implementing MPI-IO portably and with high performance

R Thakur, W Gropp, E Lusk - Proceedings of the sixth workshop on I/O in …, 1999 - dl.acm.org
We discuss the issues involved in implementing MPI-IO portably on multiple machines and
file systems and also achieving high performance. One way to implement MPI-IO portably is …

24/7 characterization of petascale I/O workloads

P Carns, R Latham, R Ross, K Iskra… - … on Cluster Computing …, 2009 - ieeexplore.ieee.org
Developing and tuning computational science applications to run on extreme scale systems
are increasingly complicated processes. Challenges such as managing memory access and …

PREEMPTION RANK, 343 PREEMPTION REQUIREMENTS, 342, 343 PREEMPT, 342 RANK, 342, 348

D LIST, F HOSTNAME, LC FILE, S LOG, W SUSPEND… - Condor, 2003 - direct.mit.edu
Mega, 471 memory, 43, 54 cache, 33, 35, 44 DRAM, 44 SRAM, 44 mesh regular, 174
message box, 296 message latency, 207 message passing, 161, 237, 296 Message …

A multiplatform study of I/O behavior on petascale supercomputers

H Luu, M Winslett, W Gropp, R Ross, P Carns… - Proceedings of the 24th …, 2015 - dl.acm.org
We examine the I/O behavior of thousands of supercomputing applications" in the wild," by
analyzing the Darshan logs of over a million jobs representing a combined total of six years …

A study on data deduplication in HPC storage systems

D Meister, J Kaiser, A Brinkmann… - SC'12: Proceedings …, 2012 - ieeexplore.ieee.org
Deduplication is a storage saving technique that is highly successful in enterprise backup
environments. On a file system, a single data block might be stored multiple times across …