A/B testing, also referred to as online controlled experimentation or continuous
experimentation, is a form of hypothesis testing where two variants of a piece of software are …

Research at the intersection of machine learning, programming languages, and software
engineering has recently taken important steps in proposing learnable probabilistic models …

Programming is a powerful and ubiquitous problem-solving tool. Systems that can assist
programmers or even generate programs themselves could make programming more …

Large language models (LMs) of code have recently shown tremendous promise in
completing code and synthesizing code from natural language descriptions. However, the …

Program synthesis or code generation aims to generate a program that satisfies a problem
specification. Recent approaches using large-scale pretrained language models (LMs) have …

This paper explores the limits of the current generation of large language models for
program synthesis in general purpose programming languages. We evaluate a collection of …

The field of graph neural networks (GNNs) has seen rapid and incredible strides over the
recent years. Graph neural networks, also known as deep learning on graphs, graph …

Benchmark datasets have a significant impact on accelerating research in programming
language tasks. In this paper, we introduce CodeXGLUE, a benchmark dataset to foster …

Large pre-trained language models such as GPT-3 [10], Codex [11], and Google's language
model [7] are now capable of generating code from natural language specifications of …

In software development through integrated development environments (IDEs), code
completion is one of the most widely used features. Nevertheless, majority of integrated …