Self-supervised learning (SSL) has achieved notable success in many speech processing
tasks, but the large model size and heavy computational cost hinder the deployment …

In this paper, we introduce self-distillation and online clustering for self-supervised speech
representation learning (DinoSR) which combines masked language modeling, self …

Self-supervised learning (SSL) has led to great strides in speech processing. However, the
resources needed to train these models has become prohibitively large as they continue to …

Self-supervised speech representation learning (SSL) has shown to be effective in various
downstream tasks, but SSL models are usually large and slow. Model compression …

Compressing self-supervised models has become increasingly necessary, as self-
supervised models become larger. While previous approaches have primarily focused on …

Distilled self-supervised models have shown competitive performance and efficiency in
recent years. However, there is a lack of experience in jointly distilling multiple self …

Self-supervised learning (SSL) has allowed substantial progress in Automatic Speech
Recognition (ASR) performance in low-resource settings. In this context, it has been …

End-to-end automatic speech recognition (ASR) models have seen revolutionary quality
gains with the recent development of large-scale universal speech models (USM). However …

Self-supervised learning (SSL) has proven vital in speech and audio-related applications.
The paradigm trains a general model on unlabeled data that can later be used to solve …

Self-Supervised Learning (SSL) models have demonstrated exceptional performance in
various speech tasks, particularly in low-resource and multilingual domains. Recent works …