Towards Optimal Caching and Model Selection for Large Model Inference

B Zhu, Y Sheng, L Zheng, C Barrett… - Advances in Neural …, 2024 - proceedings.neurips.cc
Abstract Large Language Models (LLMs) and other large foundation models have achieved
impressive results, but their size exacerbates existing resource consumption and latency …

On optimal caching and model multiplexing for large model inference

B Zhu, Y Sheng, L Zheng, C Barrett, MI Jordan… - arXiv preprint arXiv …, 2023 - arxiv.org
Large Language Models (LLMs) and other large foundation models have achieved
noteworthy success, but their size exacerbates existing resource consumption and latency …

Optimization strategies for GPUs: an overview of architectural approaches

A Masola, N Capodieci - International Journal of Parallel, Emergent …, 2023 - Taylor & Francis
Modern Cyber Physical Systems (CPS) applications require hardware capable of optimized
performance-per-watt efficency. This is usually obtained through massively parallel …

Cache memory: an analysis on performance issues

A Alsharef, P Jain, M Arora, SR Zahra… - 2021 8th international …, 2021 - ieeexplore.ieee.org
Cache memory is mainly inculcated in systems to overcome the gap created in-between the
main memory and CPUs due to their performance issues. Since, the speed of the processors …

Prediction of computer type using benchmark scores of hardware units

YS Taspinar, I Cinar, M Koklu - Selcuk University Journal of …, 2021 - sujes.selcuk.edu.tr
Users need an expert opinion to learn about their current computer or purchasing. In
addition to these, computer and computer component manufacturers have to carry out …

Meta-learning in healthcare: A survey

A Rafiei, R Moore, S Jahromi, F Hajati… - SN Computer …, 2024 - Springer
As a subset of machine learning, meta-learning, or learning to learn, aims at improving the
model's capabilities by employing prior knowledge and experience. A meta-learning …

Efficient Prompt Caching via Embedding Similarity

H Zhu, B Zhu, J Jiao - arXiv preprint arXiv:2402.01173, 2024 - arxiv.org
Large language models (LLMs) have achieved huge success in numerous natural language
process (NLP) tasks. However, it faces the challenge of significant resource consumption …

MemHC: an optimized GPU memory management framework for accelerating many-body correlation

Q Wang, Z Peng, B Ren, J Chen… - ACM Transactions on …, 2022 - dl.acm.org
The many-body correlation function is a fundamental computation kernel in modern physics
computing applications, eg, Hadron Contractions in Lattice quantum chromodynamics …

[HTML][HTML] A distributed real-time recommender system for big data streams

H Hazem, A Awad, AH Yousef - Ain Shams Engineering Journal, 2023 - Elsevier
Recommender Systems (RS) play a crucial role in our lives. As users become continuously
connected to the internet, they are less tolerant of obsolete recommendations made by an …

Implementation of LRU replacement policy for reconfigurable cache memory using FPGA

SS Omran, IA Amory - 2018 International Conference on …, 2018 - ieeexplore.ieee.org
Cache memory is an important part in computer systems. In set associative cache memory
each incoming memory block from the main memory into cache memory should be placed in …