A systematic survey and critical review on evaluating large language models: Challenges, limitations, and recommendations
Abstract Large Language Models (LLMs) have recently gained significant attention due to
their remarkable capabilities in performing diverse tasks across various domains. However …
their remarkable capabilities in performing diverse tasks across various domains. However …
Data augmentation using llms: Data perspectives, learning paradigms and challenges
In the rapidly evolving field of large language models (LLMs), data augmentation (DA) has
emerged as a pivotal technique for enhancing model performance by diversifying training …
emerged as a pivotal technique for enhancing model performance by diversifying training …
Don't make your llm an evaluation benchmark cheater
Large language models~(LLMs) have greatly advanced the frontiers of artificial intelligence,
attaining remarkable improvement in model capacity. To assess the model performance, a …
attaining remarkable improvement in model capacity. To assess the model performance, a …
How much are llms contaminated? a comprehensive survey and the llmsanitize library
With the rise of Large Language Models (LLMs) in recent years, new opportunities are
emerging, but also new challenges, and contamination is quickly becoming critical …
emerging, but also new challenges, and contamination is quickly becoming critical …
Solar 10.7 b: Scaling large language models with simple yet effective depth up-scaling
We introduce SOLAR 10.7 B, a large language model (LLM) with 10.7 billion parameters,
demonstrating superior performance in various natural language processing (NLP) tasks …
demonstrating superior performance in various natural language processing (NLP) tasks …
International Scientific Report on the Safety of Advanced AI (Interim Report)
Y Bengio, S Mindermann, D Privitera… - arXiv preprint arXiv …, 2024 - arxiv.org
This is the interim publication of the first International Scientific Report on the Safety of
Advanced AI. The report synthesises the scientific understanding of general-purpose AI--AI …
Advanced AI. The report synthesises the scientific understanding of general-purpose AI--AI …
Investigating data contamination in modern benchmarks for large language models
Recent observations have underscored a disparity between the inflated benchmark scores
and the actual performance of LLMs, raising concerns about potential contamination of …
and the actual performance of LLMs, raising concerns about potential contamination of …
One thousand and one pairs: A" novel" challenge for long-context language models
Synthetic long-context LLM benchmarks (eg," needle-in-the-haystack") test only surface-
level retrieval capabilities, but how well can long-context LLMs retrieve, synthesize, and …
level retrieval capabilities, but how well can long-context LLMs retrieve, synthesize, and …
The information retrieval experiment platform
We integrate irdatasets, ir_measures, and PyTerrier with TIRA in the Information Retrieval
Experiment Platform (TIREx) to promote more standardized, reproducible, scalable, and …
Experiment Platform (TIREx) to promote more standardized, reproducible, scalable, and …
Same task, more tokens: the impact of input length on the reasoning performance of large language models
This paper explores the impact of extending input lengths on the capabilities of Large
Language Models (LLMs). Despite LLMs advancements in recent times, their performance …
Language Models (LLMs). Despite LLMs advancements in recent times, their performance …