We introduce Differential Performance Evaluation (DPE), a framework designed to reliably
evaluate Large Language Models (LLMs) for efficient code generation. Traditional coding …

Large language models for code (ie, code LLMs) have shown strong code understanding
and generation capabilities. To evaluate the capabilities of code LLMs in various aspects …

Large Language models have achieved impressive performance in automated software
engineering. Extensive efforts have been made to evaluate the abilities of code LLMs in …

The success of AI models relies on the availability of large, diverse, and high-quality
datasets, which can be challenging to obtain due to data scarcity, privacy concerns, and …

Large Language Models (LLMs) applied to code-related applications have emerged as a
prominent field, attracting significant interest from both academia and industry. However, as …

Reasoning about asynchronous plans is challenging since it requires sequential and
parallel planning to optimize time costs. Can large language models (LLMs) succeed at this …

Test flakiness, a non-deterministic behavior of builds irrelevant to code changes, is a major
and continuing impediment to deliver-ing reliable software. The very few techniques for the …

Code generation, the task of producing source code from prompts, has seen significant
advancements with the advent of pre-trained large language models (PLMs). Despite these …

Heat exchangers (HEs) are essential in process industries for efficient thermal energy
transfer. Their design and optimization are crucial for improving energy efficiency, reducing …

Existing works have established multiple benchmarks to highlight the security risks
associated with Code GenAI. These risks are primarily reflected in two areas: a model …