We propose masked particle modeling (MPM) as a self-supervised method for learning
generic, transferable, and reusable representations on unordered sets of inputs for use in …

We introduce generative infinite-vocabulary transformers (GIVT) which generate vector
sequences with real-valued entries, instead of discrete tokens from a finite vocabulary. To …

Vector quantization, renowned for its unparalleled feature compression capabilities, has
been a prominent topic in signal processing and machine learning research for several …

Codebook collapse is a common problem in training deep generative models with discrete
representation spaces like Vector Quantized Variational Autoencoders (VQ-VAEs). We …

Pre-training large models on vast amounts of web data has proven to be an effective
approach for obtaining powerful, general models in several domains, including language …

Similar to natural language models, pre-trained genome language models are proposed to
capture the underlying intricacies within genomes with unsupervised sequence modeling …

Implicit neural representations (INR) excel in encoding videos within neural networks,
showcasing promise in computer vision tasks like video compression and denoising. INR …

Foundation models are multi-dataset and multi-task machine learning methods that once pre-
trained can be fine-tuned for a large variety of downstream applications. The successful …

Representation learning and\emph {de novo} generation of proteins are pivotal
computational biology tasks. Whilst natural language processing (NLP) techniques have …

The Large Language models (LLMs) have demonstrated supreme capabilities in text
understanding and generation, but cannot be directly applied to cross-modal tasks without …