A survey on the memory mechanism of large language model based agents

Z Zhang, X Bo, C Ma, R Li, X Chen, Q Dai, J Zhu… - arXiv preprint arXiv …, 2024 - arxiv.org
Large language model (LLM) based agents have recently attracted much attention from the
research and industry communities. Compared with original LLMs, LLM-based agents are …

Graphinsight: Unlocking insights in large language models for graph structure understanding

Y Cao, S Han, Z Gao, Z Ding, X Xie… - arXiv preprint arXiv …, 2024 - arxiv.org
Although Large Language Models (LLMs) have demonstrated potential in processing
graphs, they struggle with comprehending graphical structure information through prompts …

Timer-XL: Long-Context Transformers for Unified Time Series Forecasting

Y Liu, G Qin, X Huang, J Wang, M Long - arXiv preprint arXiv:2410.04803, 2024 - arxiv.org
We present Timer-XL, a generative Transformer for unified time series forecasting. To
uniformly predict 1D and 2D time series, we generalize next token prediction, predominantly …

Infinipot: Infinite context processing on memory-constrained llms

M Kim, K Shim, J Choi, S Chang - arXiv preprint arXiv:2410.01518, 2024 - arxiv.org
Handling long input contexts remains a significant challenge for Large Language Models
(LLMs), particularly in resource-constrained environments such as mobile devices. Our work …

Self-Updatable Large Language Models with Parameter Integration

Y Wang, X Liu, X Chen, S O'Brien, J Wu… - arXiv preprint arXiv …, 2024 - arxiv.org
Despite significant advancements in large language models (LLMs), the rapid and frequent
integration of small-scale experiences, such as interactions with surrounding objects …

PCToolkit: A Unified Plug-and-Play Prompt Compression Toolkit of Large Language Models

J Li, Y Lan, L Wang, H Wang - arXiv preprint arXiv:2403.17411, 2024 - arxiv.org
Prompt compression is an innovative method for efficiently condensing input prompts while
preserving essential information. To facilitate quick-start services, user-friendly interfaces …

On the token distance modeling ability of higher RoPE attention dimension

X Hong, C Jiang, B Qi, F Meng, M Yu, B Zhou… - arXiv preprint arXiv …, 2024 - arxiv.org
Length extrapolation algorithms based on Rotary position embedding (RoPE) have shown
promising results in extending the context length of language models. However …

HyQE: Ranking Contexts with Hypothetical Query Embeddings

W Zhou, J Zhang, H Hasson, A Singh, W Li - arXiv preprint arXiv …, 2024 - arxiv.org
In retrieval-augmented systems, context ranking techniques are commonly employed to
reorder the retrieved contexts based on their relevance to a user query. A standard …

Towards LifeSpan Cognitive Systems

Y Wang, C Han, T Wu, X He, W Zhou, N Sadeq… - arXiv preprint arXiv …, 2024 - arxiv.org
Building a human-like system that continuously interacts with complex environments--
whether simulated digital worlds or human society--presents several key challenges. Central …

Large Language Models for Constructing and Optimizing Machine Learning Workflows: A Survey

Y Gu, H You, J Cao, M Yu - arXiv preprint arXiv:2411.10478, 2024 - arxiv.org
Building effective machine learning (ML) workflows to address complex tasks is a primary
focus of the Automatic ML (AutoML) community and a critical step toward achieving artificial …