This survey paper provides a comprehensive overview of the recent advancements and
challenges in applying large language models to the field of audio signal processing. Audio …

Neural audio codecs are initially introduced to compress audio data into compact codes to
reduce transmission latency. Researchers recently discovered the potential of codecs as …

We propose a decoder-only language model, VoxtLM, that can perform four tasks: speech
recognition, speech synthesis, text generation, and speech continuation. VoxtLM integrates …

We consider the task of generating diverse and realistic videos guided by natural audio
samples from a wide variety of semantic classes. For this task, the videos are required to be …

Large language models (LLMs) have gained considerable attention for Artificial Intelligence
Generated Content (AIGC), particularly with the emergence of ChatGPT. However, the direct …

This paper introduces a novel voice conversion (VC) model, guided by text instructions such
as “articulate slowly with a deep tone “or “speak in a cheerful boyish voice”. Unlike …

Ever since the development of GPT-3 in the natural language processing (NLP) field, in-
context learning (ICL) has played an important role in utilizing large language models …

We introduce SPIRIT-LM, a foundation multimodal language model that freely mixes text and
speech. Our model is based on a pretrained text language model that we extend to the …

Prompting has become a practical method for utilizing pre-trained language models (LMs).
This approach offers several advantages. It allows an LM to adapt to new tasks with minimal …

Prompting and adapter tuning have emerged as efficient alternatives to fine-tuning (FT)
methods. However, existing studies on speech prompting focused on classification tasks …