Multimodal large language models are generalist medical image interpreters

T Han, LC Adams, S Nebelung, JN Kather, KK Bressem… - medRxiv, 2023 - medrxiv.org
Advanced multimodal large language models (LLM), such as GPT-4V (ision) and Gemini
Ultra, have shown promising results in the diagnosis of complex pathological conditions …

Simplifying Multimodality: Unimodal Approach to Multimodal Challenges in Radiology with General-Domain Large Language Model

S Cho, C Kim, J Lee, C Chilkunda, S Choi… - arXiv preprint arXiv …, 2024 - arxiv.org
Recent advancements in Large Multimodal Models (LMMs) have attracted interest in their
generalization capability with only a few samples in the prompt. This progress is particularly …

Evaluating General Vision-Language Models for Clinical Medicine

Y Jiang, JA Omiye, C Zakka, M Moor, H Gui, S Alipour… - medRxiv, 2024 - medrxiv.org
Recently emerging large multimodal models (LMMs) utilize various types of data modalities,
including text and visual inputs to generate outputs. The incorporation of LMMs into clinical …

Comparative Analysis of GPT-4Vision, GPT-4 and Open Source LLMs in Clinical Diagnostic Accuracy: A Benchmark Against Human Expertise

T Han, LC Adams, K Bressem, F Busch, L Huck… - medRxiv, 2023 - medrxiv.org
Importance Medicine is poised for transformation with artificial general intelligence
becoming integral to almost all clinical environments. Currently, the performance of …

Can gpt-4v (ision) serve medical applications? case studies on gpt-4v for multimodal medical diagnosis

C Wu, J Lei, Q Zheng, W Zhao, W Lin, X Zhang… - arXiv preprint arXiv …, 2023 - arxiv.org
Driven by the large foundation models, the development of artificial intelligence has
witnessed tremendous progress lately, leading to a surge of general interest from the public …

A Generalist Learner for Multifaceted Medical Image Interpretation

HY Zhou, S Adithan, JN Acosta, EJ Topol… - arXiv preprint arXiv …, 2024 - arxiv.org
Current medical artificial intelligence systems are often limited to narrow applications,
hindering their widespread adoption in clinical practice. To address this limitation, we …

Evaluating LLM--Generated Multimodal Diagnosis from Medical Images and Symptom Analysis

DP Panagoulias, M Virvou, GA Tsihrintzis - arXiv preprint arXiv …, 2024 - arxiv.org
Large language models (LLMs) constitute a breakthrough state-of-the-art Artificial
Intelligence technology which is rapidly evolving and promises to aid in medical diagnosis …

Holistic evaluation of gpt-4v for biomedical imaging

Z Liu, H Jiang, T Zhong, Z Wu, C Ma, Y Li, X Yu… - arXiv preprint arXiv …, 2023 - arxiv.org
In this paper, we present a large-scale evaluation probing GPT-4V's capabilities and
limitations for biomedical image analysis. GPT-4V represents a breakthrough in artificial …

HuatuoGPT-Vision, Towards Injecting Medical Visual Knowledge into Multimodal LLMs at Scale

J Chen, R Ouyang, A Gao, S Chen, GH Chen… - arXiv preprint arXiv …, 2024 - arxiv.org
The rapid development of multimodal large language models (MLLMs), such as GPT-4V,
has led to significant advancements. However, these models still face challenges in medical …

Large language models illuminate a progressive pathway to artificial healthcare assistant: A review

M Yuan, P Bao, J Yuan, Y Shen, Z Chen, Y Xie… - arXiv preprint arXiv …, 2023 - arxiv.org
With the rapid development of artificial intelligence, large language models (LLMs) have
shown promising capabilities in mimicking human-level language comprehension and …