Mixture of parrots: Experts improve memorization more than reasoning
The Mixture-of-Experts (MoE) architecture enables a significant increase in the total number
of model parameters with minimal computational overhead. However, it is not clear what …
of model parameters with minimal computational overhead. However, it is not clear what …
Reasoning in large language models: A geometric perspective
R Cosentino, S Shekkizhar - arXiv preprint arXiv:2407.02678, 2024 - arxiv.org
The advancement of large language models (LLMs) for real-world applications hinges
critically on enhancing their reasoning capabilities. In this work, we explore the reasoning …
critically on enhancing their reasoning capabilities. In this work, we explore the reasoning …
A theory for compressibility of graph transformers for transductive learning
Transductive tasks on graphs differ fundamentally from typical supervised machine learning
tasks, as the independent and identically distributed (iid) assumption does not hold among …
tasks, as the independent and identically distributed (iid) assumption does not hold among …
Lost-in-Distance: Impact of Contextual Proximity on LLM Performance in Graph Tasks
Despite significant advancements, Large Language Models (LLMs) exhibit blind spots that
impair their ability to retrieve and process relevant contextual data effectively. We …
impair their ability to retrieve and process relevant contextual data effectively. We …
The CLRS-Text Algorithmic Reasoning Language Benchmark
Eliciting reasoning capabilities from language models (LMs) is a critical direction on the path
towards building intelligent systems. Most recent studies dedicated to reasoning focus on …
towards building intelligent systems. Most recent studies dedicated to reasoning focus on …
Graph Reasoning with LLMs (GReaL)
Graphs are a powerful tool for representing and analyzing complex relationships in real-
world applications. Large Language Models (LLMs) have demonstrated impressive …
world applications. Large Language Models (LLMs) have demonstrated impressive …
Text-space Graph Foundation Models: Comprehensive Benchmarks and New Insights
Given the ubiquity of graph data and its applications in diverse domains, building a Graph
Foundation Model (GFM) that can work well across different graphs and tasks with a unified …
Foundation Model (GFM) that can work well across different graphs and tasks with a unified …
GraphTeam: Facilitating Large Language Model-based Graph Analysis via Multi-Agent Collaboration
Graphs are widely used for modeling relational data in real-world scenarios, such as social
networks and urban computing. Existing LLM-based graph analysis approaches either …
networks and urban computing. Existing LLM-based graph analysis approaches either …
Generalizing from SIMPLE to HARD Visual Reasoning: Can We Mitigate Modality Imbalance in VLMs?
While Vision Language Models (VLMs) are impressive in tasks such as visual question
answering (VQA) and image captioning, their ability to apply multi-step reasoning to images …
answering (VQA) and image captioning, their ability to apply multi-step reasoning to images …
Test of Time: A Benchmark for Evaluating LLMs on Temporal Reasoning
Large language models (LLMs) have showcased remarkable reasoning capabilities, yet
they remain susceptible to errors, particularly in temporal reasoning tasks involving complex …
they remain susceptible to errors, particularly in temporal reasoning tasks involving complex …