Pre-trained language models in biomedical domain: A systematic survey

B Wang, Q Xie, J Pei, Z Chen, P Tiwari, Z Li… - ACM Computing …, 2023 - dl.acm.org
Pre-trained language models (PLMs) have been the de facto paradigm for most natural
language processing tasks. This also benefits the biomedical domain: researchers from …

Language model behavior: A comprehensive survey

TA Chang, BK Bergen - Computational Linguistics, 2024 - direct.mit.edu
Transformer language models have received widespread public attention, yet their
generated text is often surprising even to NLP researchers. In this survey, we discuss over …

When not to trust language models: Investigating effectiveness of parametric and non-parametric memories

A Mallen, A Asai, V Zhong, R Das, D Khashabi… - arXiv preprint arXiv …, 2022 - arxiv.org
Despite their impressive performance on diverse tasks, large language models (LMs) still
struggle with tasks requiring rich world knowledge, implying the limitations of relying solely …

Can we edit factual knowledge by in-context learning?

C Zheng, L Li, Q Dong, Y Fan, Z Wu, J Xu… - arXiv preprint arXiv …, 2023 - arxiv.org
Previous studies have shown that large language models (LLMs) like GPTs store massive
factual knowledge in their parameters. However, the stored knowledge could be false or out …

Pmet: Precise model editing in a transformer

X Li, S Li, S Song, J Yang, J Ma, J Yu - Proceedings of the AAAI …, 2024 - ojs.aaai.org
Model editing techniques modify a minor proportion of knowledge in Large Language
Models (LLMs) at a relatively low cost, which have demonstrated notable success. Existing …

Calibrating factual knowledge in pretrained language models

Q Dong, D Dai, Y Song, J Xu, Z Sui, L Li - arXiv preprint arXiv:2210.03329, 2022 - arxiv.org
Previous literature has proved that Pretrained Language Models (PLMs) can store factual
knowledge. However, we find that facts stored in the PLMs are not always correct. It …

Chatgpt is not enough: Enhancing large language models with knowledge graphs for fact-aware language modeling

L Yang, H Chen, Z Li, X Ding, X Wu - arXiv preprint arXiv:2306.11489, 2023 - arxiv.org
Recently, ChatGPT, a representative large language model (LLM), has gained considerable
attention due to its powerful emergent abilities. Some researchers suggest that LLMs could …

DeepStruct: Pretraining of language models for structure prediction

C Wang, X Liu, Z Chen, H Hong, J Tang… - arXiv preprint arXiv …, 2022 - arxiv.org
We introduce a method for improving the structural understanding abilities of language
models. Unlike previous approaches that finetune the models with task-specific …

Large language models and knowledge graphs: Opportunities and challenges

JZ Pan, S Razniewski, JC Kalo, S Singhania… - arXiv preprint arXiv …, 2023 - arxiv.org
Large Language Models (LLMs) have taken Knowledge Representation--and the world--by
storm. This inflection point marks a shift from explicit knowledge representation to a renewed …

Who answers it better? an in-depth analysis of chatgpt and stack overflow answers to software engineering questions

S Kabir, DN Udo-Imeh, B Kou, T Zhang - arXiv preprint arXiv:2308.02312, 2023 - arxiv.org
Q&A platforms have been an integral part of the web-help-seeking behavior of programmers
over the past decade. However, with the recent introduction of ChatGPT, the paradigm of …