The llama 3 herd of models
Modern artificial intelligence (AI) systems are powered by foundation models. This paper
presents a new set of foundation models, called Llama 3. It is a herd of language models …
presents a new set of foundation models, called Llama 3. It is a herd of language models …
Cogagent: A visual language model for gui agents
People are spending an enormous amount of time on digital devices through graphical user
interfaces (GUIs) eg computer or smartphone screens. Large language models (LLMs) such …
interfaces (GUIs) eg computer or smartphone screens. Large language models (LLMs) such …
Monkey: Image resolution and text label are important things for large multi-modal models
Z Li, B Yang, Q Liu, Z Ma, S Zhang… - Proceedings of the …, 2024 - openaccess.thecvf.com
Abstract Large Multimodal Models (LMMs) have shown promise in vision-language tasks but
struggle with high-resolution input and detailed scene understanding. Addressing these …
struggle with high-resolution input and detailed scene understanding. Addressing these …
Webarena: A realistic web environment for building autonomous agents
With advances in generative AI, there is now potential for autonomous agents to manage
daily tasks via natural language commands. However, current agents are primarily created …
daily tasks via natural language commands. However, current agents are primarily created …
Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts
Although Large Language Models (LLMs) and Large Multimodal Models (LMMs) exhibit
impressive skills in various domains, their ability for mathematical reasoning within visual …
impressive skills in various domains, their ability for mathematical reasoning within visual …
Self-supervised multimodal learning: A survey
Multimodal learning, which aims to understand and analyze information from multiple
modalities, has achieved substantial progress in the supervised regime in recent years …
modalities, has achieved substantial progress in the supervised regime in recent years …
Pali-x: On scaling up a multilingual vision and language model
We present the training recipe and results of scaling up PaLI-X, a multilingual vision and
language model, both in terms of size of the components and the breadth of its training task …
language model, both in terms of size of the components and the breadth of its training task …
Image captioners are scalable vision learners too
Contrastive pretraining on image-text pairs from the web is one of the most popular large-
scale pretraining strategies for vision backbones, especially in the context of large …
scale pretraining strategies for vision backbones, especially in the context of large …
From pixels to ui actions: Learning to follow instructions via graphical user interfaces
Much of the previous work towards digital agents for graphical user interfaces (GUIs) has
relied on text-based representations (derived from HTML or other structured data sources) …
relied on text-based representations (derived from HTML or other structured data sources) …
mplug-docowl: Modularized multimodal large language model for document understanding
Document understanding refers to automatically extract, analyze and comprehend
information from various types of digital documents, such as a web page. Existing Multi …
information from various types of digital documents, such as a web page. Existing Multi …