Distributed storage optimization using multi-agent systems in Hadoop

M Sais, N Rafalia, R Mahdaoui… - E3S Web of …, 2023 - e3s-conferences.org
Understanding data and extracting information from it are the main objectives of data
science, especially when it comes to big data. To achieve these goals, it is necessary to …

[PDF][PDF] An intelligent storage optimization technique for heterogeneous hadoop clusters

NSK Chakravarthy, N Sudhakar… - International Journal of …, 2018 - academia.edu
Big data storage and processing can be carried out with the help of Hadoop technology.
Hadoop distributed file system store data using chunks, whose size is multiple of 64MB. The …

HDFS optimization strategy based on hierarchical storage of hot and cold data

Y Guan, Z Ma, L Li - Procedia CIRP, 2019 - Elsevier
HDFS has been the core storage service of Hadoop big data platform for many years, with
the continuous development of deep learning, artificial intelligence and other technologies …

Heterogeneity-aware data placement in Hybrid Clouds

JD Marquez, JD Gonzalez, OH Mondragon - International Conference on …, 2019 - Springer
In next-generation cloud computing clusters, performance of data-intensive applications will
be limited, among other factors, by disks data transfer rates. In order to mitigate performance …

A Comparative Study of Indexing Techniques Effect in Big Data System Storage Optimization

MM Hamad - 2020 2nd Al-Noor International Conference for …, 2020 - ieeexplore.ieee.org
The growth of the information and networking technologies such as mobile devices, social
media, and cloud applications required an efficient management big data system. Moreover …

Swarm Intelligence with a Chaotic Leader and a Salp algorithm: HDFS optimization for reduced latency and enhanced availability

N Jagadish Kumar, D Dhinakaran… - Concurrency and … - Wiley Online Library
The Hadoop distributed file system (HDFS) effectively manages data by segmenting it into
blocks distributed across DataNodes in its cluster. While default block sizes in Hadoop 2. x …

[PDF][PDF] New data placement strategy in the HADOOP framework

A ELomari, L Hassouni, A MAIZATE - International Journal of …, 2021 - academia.edu
Today, the data quantities generated and exchanged between information systems
continues to increase. Storing and exploiting such quantities require can't be done without …

The HDFS replica placement policies: A comparative experimental investigation

RWA Fazul, PP Barcelos - IFIP International Conference on Distributed …, 2022 - Springer
Abstract The Hadoop Distributed File System (HDFS) is a robust and flexible file system
designed for reliably storing large volumes of data in distributed environments. Its storage …

Optimizing Read Operations of Hadoop Distributed File System on Heterogeneous Storages.

J Lee, J Lee, SW Lee - Journal of Information Science & …, 2021 - search.ebscohost.com
The key challenge in big data processing frameworks such as Hadoop distributed file
system (HDFS) is to optimize the throughput for read operations. Toward this goal, several …

[PDF][PDF] An efficient dynamic data replication for HDFS using erasure coding

F John, S Gopinath, E Sherly - International Journal of Computer …, 2017 - academia.edu
The Hadoop Distributed File System (HDFS) component of Apache Hadoop helps in
distributed storage of big data with a cluster of commodity hardware. HDFS ensures …