With the dramatic increase in internet users and their demand for real-time network
performance, Spark has distributed computing environment has emerged. It is widely used …

Spark is widely used due to its high performance caching mechanism and high scalability,
which still causes uneven workloads and produces useless intermediate caching results …

Caching is one of the most important techniques for the popular distributed big data
processing framework Spark. For this big data parallel computing framework, which is …

Spark is a distributed data processing framework based on memory. Memory allocation is a
focus question of Spark research. A good memory allocation scheme can effectively improve …

Spark is currently the most widely used distributed computing framework, and its key data
abstraction concept, Resilient Distributed Dataset (RDD), brings significant performance …

Improper data replacement and inappropriate selection of job scheduling policy are
important reasons for the degradation of Spark system operation speed, which directly …

Apache Spark caches reusable data into memory/disk. From our preliminary evaluation, we
have found that a memory-and-disk caching is ineffective compared to disk-only caching …

Both data shuffling and cache recovery are essential parts of the Spark system, and they
directly affect Spark parallel computing performance. Existing dynamic partitioning schemes …

In the era of Big Data, processing large amounts of data through data-intensive applications,
is presenting a challenge. An in-memory distributed computing system; Apache Spark is …

As a typical representative of distributed computing framework, Spark has been continuously
developed and popularized. It reduces the data transmission time through efficient memory …