Lever: Learning to verify language-to-code generation with execution

A Ni, S Iyer, D Radev, V Stoyanov… - International …, 2023 - proceedings.mlr.press
The advent of large language models trained on code (code LLMs) has led to significant
progress in language-to-code generation. State-of-the-art approaches in this area combine …

Llm-assisted code cleaning for training accurate code generators

N Jain, T Zhang, WL Chiang, JE Gonzalez… - arXiv preprint arXiv …, 2023 - arxiv.org
Natural language to code generation is an important application area of LLMs and has
received wide attention from the community. The majority of relevant studies have …

Is your code generated by chatgpt really correct? rigorous evaluation of large language models for code generation

J Liu, CS Xia, Y Wang, L Zhang - Advances in Neural …, 2024 - proceedings.neurips.cc
Program synthesis has been long studied with recent approaches focused on directly using
the power of Large Language Models (LLMs) to generate code. Programming benchmarks …

Monitor-guided decoding of code LMs with static analysis of repository context

LA Agrawal, A Kanade, N Goyal… - Advances in Neural …, 2024 - proceedings.neurips.cc
Abstract Language models of code (LMs) work well when the surrounding code provides
sufficient context. This is not true when it becomes necessary to use types, functionality or …

Natural language to code translation with execution

F Shi, D Fried, M Ghazvininejad, L Zettlemoyer… - arXiv preprint arXiv …, 2022 - arxiv.org
Generative models of code, pretrained on large corpora of programs, have shown great
success in translating natural language to code (Chen et al., 2021; Austin et al., 2021; Li et …

A study on robustness and reliability of large language model code generation

L Zhong, Z Wang - arXiv preprint arXiv:2308.10335, 2023 - arxiv.org
Recently, the large language models (LLMs) have shown extraordinary ability in
understanding natural language and generating programming code. It has been a common …

Codet5+: Open code large language models for code understanding and generation

Y Wang, H Le, AD Gotmare, NDQ Bui, J Li… - arXiv preprint arXiv …, 2023 - arxiv.org
Large language models (LLMs) pretrained on vast source code have achieved prominent
progress in code intelligence. However, existing code LLMs have two main limitations in …

Teaching large language models to self-debug

X Chen, M Lin, N Schärli, D Zhou - arXiv preprint arXiv:2304.05128, 2023 - arxiv.org
Large language models (LLMs) have achieved impressive performance on code generation.
However, for complex programming tasks, generating the correct solution in one go …

Algo: Synthesizing algorithmic programs with generated oracle verifiers

K Zhang, D Wang, J Xia… - Advances in Neural …, 2023 - proceedings.neurips.cc
Large language models (LLMs) excel at implementing code from functionality descriptions
but struggle with algorithmic problems that require not only implementation but also …

Codescore: Evaluating code generation by learning code execution

Y Dong, J Ding, X Jiang, G Li, Z Li, Z Jin - arXiv preprint arXiv:2301.09043, 2023 - arxiv.org
A proper code evaluation metric (CEM) profoundly impacts the evolution of code generation,
which is an important research field in NLP and software engineering. Prevailing match …