BioBERTurk: Exploring Turkish Biomedical Language Model Development Strategies in Low-Resource Setting

H Türkmen, O Dikenelli, C Eraslan, MC Callı… - Journal of Healthcare …, 2023 - Springer
Pretrained language models augmented with in-domain corpora show impressive results in
biomedicine and clinical Natural Language Processing (NLP) tasks in English. However …

Developing pretrained language models for turkish biomedical domain

H Türkmen, O Dikenelli, C Eraslan… - 2022 IEEE 10th …, 2022 - ieeexplore.ieee.org
Pretrained language models elevated with in-domain corpora show impressive results in
biomedicine and clinical NLP tasks in English. However, there is minimal work in low …

Harnessing the power of BERT in the Turkish clinical domain: pretraining approaches for limited data scenarios

H Türkmen, O Dikenelli, C Eraslan, MC Çallı… - arXiv preprint arXiv …, 2023 - arxiv.org
In recent years, major advancements in natural language processing (NLP) have been
driven by the emergence of large language models (LLMs), which have significantly …

Pre-trained language models in medicine: A survey

X Luo, Z Deng, B Yang, MY Luo - Artificial Intelligence in Medicine, 2024 - Elsevier
With the rapid progress in Natural Language Processing (NLP), Pre-trained Language
Models (PLM) such as BERT, BioBERT, and ChatGPT have shown great potential in various …

Pre-trained language models in biomedical domain: A systematic survey

B Wang, Q Xie, J Pei, Z Chen, P Tiwari, Z Li… - ACM Computing …, 2023 - dl.acm.org
Pre-trained language models (PLMs) have been the de facto paradigm for most natural
language processing tasks. This also benefits the biomedical domain: researchers from …

A clinical specific BERT developed using a huge Japanese clinical text corpus

Y Kawazoe, D Shibata, E Shinohara, E Aramaki, K Ohe - Plos one, 2021 - journals.plos.org
Generalized language models that are pre-trained with a large corpus have achieved great
performance on natural language tasks. While many pre-trained transformers for English are …

Domain-specific language model pretraining for biomedical natural language processing

Y Gu, R Tinn, H Cheng, M Lucas, N Usuyama… - ACM Transactions on …, 2021 - dl.acm.org
Pretraining large neural language models, such as BERT, has led to impressive gains on
many natural language processing (NLP) tasks. However, most pretraining efforts focus on …

Pretrained biomedical language models for clinical NLP in Spanish

CP Carrino, J Llop, M Pàmies… - Proceedings of the …, 2022 - aclanthology.org
This work presents the first large-scale biomedical Spanish language models trained from
scratch, using large biomedical corpora consisting of a total of 1.1 B tokens and an EHR …

MediGPT: Exploring Potentials of Conventional and Large Language Models on Medical Data

MAT Rony, MS Islam, T Sultan, S Alshathri… - IEEE …, 2024 - ieeexplore.ieee.org
Medical text classification organizes medical documents into categories to streamline
information retrieval and support clinical decision-making. Traditional machine learning …

Medical mT5: an open-source multilingual text-to-text LLM for the medical domain

I García-Ferrero, R Agerri, AA Salazar, E Cabrio… - arXiv preprint arXiv …, 2024 - arxiv.org
Research on language technology for the development of medical applications is currently a
hot topic in Natural Language Understanding and Generation. Thus, a number of large …