Predictive performance modeling for distributed batch processing using black box monitoring and machine learning

C Witt, M Bux, W Gusew, U Leser - Information Systems, 2019 - Elsevier
In many domains, the previous decade was characterized by increasing data volumes and
growing complexity of data analyses, creating new demands for batch processing on …

Efficient memory disaggregation with infiniswap

J Gu, Y Lee, Y Zhang, M Chowdhury… - 14th USENIX Symposium …, 2017 - usenix.org
Memory-intensive applications suffer large performance loss when their working sets do not
fully fit in memory. Yet, they cannot leverage otherwise unused remote memory when paging …

Caerus:{NIMBLE} task scheduling for serverless analytics

H Zhang, Y Tang, A Khandelwal, J Chen… - 18th USENIX Symposium …, 2021 - usenix.org
Serverless platforms facilitate transparent resource elasticity and fine-grained billing, making
them an attractive choice for data analytics. We find that while server-centric analytics …

Black or white? how to develop an autotuner for memory-based analytics

M Kunjir, S Babu - Proceedings of the 2020 ACM SIGMOD International …, 2020 - dl.acm.org
There is a lot of interest today in building autonomous (or, self-driving) data processing
systems. An emerging school of thought is to leverage AI-driven" black box" algorithms for …

Memory disaggregation: Research problems and opportunities

L Liu, W Cao, S Sahin, Q Zhang… - 2019 IEEE 39th …, 2019 - ieeexplore.ieee.org
Memory usage imbalance has been consistently observed in many virtualized Clouds and
production datacenters. Such temporal memory utilization variance is a major root cause for …

Hierarchical orchestration of disaggregated memory

W Cao, L Liu - IEEE Transactions on Computers, 2020 - ieeexplore.ieee.org
This article presents XMemPod, a hierarchical disaggregated memory orchestration system.
XMemPod virtualizes cluster wide memory to scale large memory workloads in virtualized …

CHASE: Accelerating Distributed Pointer-Traversals on Disaggregated Memory

Y Tang, S Lee, A Khandelwal - arXiv preprint arXiv:2305.02388, 2023 - arxiv.org
Caches at CPU nodes in disaggregated memory architectures amortize the high data
access latency over the network. However, such caches are fundamentally unable to …

Performance prediction for convolutional neural network on spark cluster

R Myung, H Yu - Electronics, 2020 - mdpi.com
Applications with large-scale data are processed on a distributed system, such as Spark, as
they are data-and computation-intensive. Predicting the performance of such applications is …

In-memory dataflow execution with dynamic placement of cache operations

VM Gottin, FAM Porto, YM Souto - US Patent 10,802,975, 2020 - Google Patents
A dataflow execution environment is provided with dynamic placement of cache operations.
An exemplary method comprises: obtaining a first cache placement plan for a dataflow …

Machine-learning based memory prediction model for data parallel workloads in apache spark

R Myung, S Choi - Symmetry, 2021 - mdpi.com
A lack of memory can lead to job failures or increase processing times for garbage
collection. However, if too much memory is provided, the processing time is only marginally …