High-performance and resilient key-value store with online erasure coding for big data workloads

D Shankar, X Lu, DK Panda - 2017 IEEE 37th International …, 2017 - ieeexplore.ieee.org
Distributed key-value store-based caching solutions are being increasingly used to
accelerate Big Data applications on modern HPC clusters. This has necessitated …

Erasure coding for small objects in in-memory KV storage

MMT Yiu, HHW Chan, PPC Lee - … of the 10th ACM International Systems …, 2017 - dl.acm.org
We present MemEC, an erasure-coding-based in-memory key-value (KV) store that
achieves high availability and fast recovery while keeping low data redundancy across …

RE-store: Reliable and efficient KV-store with erasure coding and replication

Y Li, J Zhou, W Wang, Y Chen - 2019 IEEE International …, 2019 - ieeexplore.ieee.org
In-memory key/value stores (KV-stores) are a key building block for numerous applications
running on a cluster. As cluster scales have grown, efficiency and availability have become …

Scalable data resilience for in-memory data staging

S Duan, P Subedi, K Teranishi, P Davis… - 2018 IEEE …, 2018 - ieeexplore.ieee.org
The dramatic increase in the scale of current and planned high-end HPC systems is leading
new challenges, such as the growing costs of data movement and IO, and the reduced mean …

Mics: Mingling chained storage combining replication and erasure coding

Y Tang, J Yin, W Lo, Y Li, S Deng… - 2015 IEEE 34th …, 2015 - ieeexplore.ieee.org
High reliability, low space cost, and efficient read/write performance are all desirable
properties for cloud storage systems. Due to the inherent conflicts, however, simultaneously …

Fault tolerance performance evaluation of large-scale distributed storage systems HDFS and Ceph case study

Y Arafa, A Barai, M Zheng… - 2018 IEEE High …, 2018 - ieeexplore.ieee.org
Large-scale distributed systems are a collection of loosely coupled computers
interconnected by a communication network. They are now an integral part of everyday life …

LogECMem: coupling erasure-coded in-memory key-value stores with parity logging

L Cheng, Y Hu, Z Ke, J Xu, Q Yao, D Feng… - Proceedings of the …, 2021 - dl.acm.org
In-memory key-value stores are often used to speed up Big Data workloads on modern HPC
clusters. To maintain their high availability, erasure coding has been recently adopted as a …

Boldio: A hybrid and resilient burst-buffer over lustre for accelerating big data i/o

D Shankar, X Lu, DK Panda - … Conference on Big Data (Big Data …, 2016 - ieeexplore.ieee.org
The limitation of local storage space in the HPC environments has placed an unprecedented
demand on the performance of the underlying shared parallel file systems. This has …

High-performance hybrid key-value store on modern clusters with RDMA interconnects and SSDs: Non-blocking extensions, designs, and benefits

D Shankar, X Lu, N Islam… - 2016 IEEE …, 2016 - ieeexplore.ieee.org
High-performance, distributed key-value store-based caching solutions, such as
Memcached, have played a crucial role in enhancing the performance of many Online and …

TEA: A traffic-efficient erasure-coded archival scheme for in-memory stores

B Xu, J Huang, Q Cao, X Qin - … of the 48th International Conference on …, 2019 - dl.acm.org
To achieve good trade-off between access performance and memory efficiency, it is
appropriate to adopt replication and erasure coding to keep popular and unpopular in …