A systematic evaluation of large language models of code

FF Xu, U Alon, G Neubig, VJ Hellendoorn - Proceedings of the 6th ACM …, 2022 - dl.acm.org
Large language models (LMs) of code have recently shown tremendous promise in
completing code and synthesizing code from natural language descriptions. However, the …

Assessing the Security of GitHub Copilot's Generated Code-A Targeted Replication Study

V Majdinasab, MJ Bishop, S Rasheed… - … on Software Analysis …, 2024 - ieeexplore.ieee.org
AI-powered code generation models have been developing rapidly, allowing developers to
expedite code generation and thus improve their productivity. These models are trained on …

Execution-based evaluation for open-domain code generation

Z Wang, S Zhou, D Fried, G Neubig - arXiv preprint arXiv:2212.10481, 2022 - arxiv.org
To extend the scope of coding queries to more realistic settings, we propose ODEX, the first
Open-Domain EXecution-based natural language (NL) to Python code generation dataset …

Mconala: a benchmark for code generation from multiple natural languages

Z Wang, G Cuenca, S Zhou, FF Xu… - arXiv preprint arXiv …, 2022 - arxiv.org
While there has been a recent burgeoning of applications at the intersection of natural and
programming languages, such as code generation and code summarization, these …

Identifying smart contract security issues in code snippets from stack overflow

J Chen, C Chen, J Hu, J Grundy, Y Wang… - Proceedings of the 33rd …, 2024 - dl.acm.org
Smart contract developers frequently seek solutions to developmental challenges on Q&A
platforms such as Stack Overflow (SO). Although community responses often provide viable …

An empirical study of oss-fuzz bugs

ZY Ding, C Le Goues - 2021 IEEE/ACM 18th International …, 2021 - ieeexplore.ieee.org
Continuous fuzzing is an increasingly popular technique for automated quality and security
assurance. Google maintains OSS-Fuzz: a continuous fuzzing service for open source …

A study of c/c++ code weaknesses on stack overflow

H Zhang, S Wang, H Li, TH Chen… - IEEE Transactions on …, 2021 - ieeexplore.ieee.org
Stack Overflow hosts millions of solutions that aim to solve developers' programming issues.
In this crowdsourced question answering process, Stack Overflow becomes a code hosting …

How have views on Software Quality differed over time? Research and practice viewpoints

IG Ndukwe, SA Licorish, A Tahir… - Journal of Systems and …, 2023 - Elsevier
Context: Over the years, there has been debate about what constitutes software quality and
how it should be measured. This controversy has caused uncertainty across the software …

Bugs in large language models generated code

F Tambon, AM Dakhel, A Nikanjam, F Khomh… - arXiv preprint arXiv …, 2024 - arxiv.org
Large Language Models (LLMs) for code have gained significant attention recently. They
can generate code in different programming languages based on provided prompts, fulfilling …

Kgsecconfig: a knowledge graph based approach for secured container orchestrator configuration

MU Haque, MM Kholoosi… - 2022 IEEE International …, 2022 - ieeexplore.ieee.org
Container Orchestrator (CO) is a vital technology for managing clusters of containers, which
may form a virtualized infrastructure for developing and operating software systems. Like …