A comprehensive survey on pretrained foundation models: A history from bert to chatgpt

C Zhou, Q Li, C Li, J Yu, Y Liu, G Wang… - arXiv preprint arXiv …, 2023 - arxiv.org
Pretrained Foundation Models (PFMs) are regarded as the foundation for various
downstream tasks with different data modalities. A PFM (eg, BERT, ChatGPT, and GPT-4) is …

Research of Chinese intangible cultural heritage knowledge graph construction and attribute value extraction with graph attention network

T Fan, H Wang - Information Processing & Management, 2022 - Elsevier
The development of digital technology promotes the construction of the Intangible cultural
heritage (ICH) database but the data is still unorganized and not linked well, which makes …

LIMIT-BERT: Linguistic informed multi-task bert

J Zhou, Z Zhang, H Zhao, S Zhang - arXiv preprint arXiv:1910.14296, 2019 - arxiv.org
In this paper, we present a Linguistic Informed Multi-Task BERT (LIMIT-BERT) for learning
language representations across multiple linguistic tasks by Multi-Task Learning (MTL) …

Investigating patterns of change, stability, and interaction among scientific disciplines using embeddings

B McGillivray, GB Jenset, K Salama… - Humanities and Social …, 2022 - nature.com
Multi-disciplinary and inter-disciplinary collaboration can be an appropriate response to
tackling the increasingly complex problems faced by today's society. Scientific disciplines …

Parsing all: Syntax and semantics, dependencies and spans

J Zhou, Z Li, H Zhao - arXiv preprint arXiv:1908.11522, 2019 - arxiv.org
Both syntactic and semantic structures are key linguistic contextual clues, in which parsing
the latter has been well shown beneficial from parsing the former. However, few works ever …

Named entity recognition and relation extraction using enhanced table filling by contextualized representations

Y Ma, T Hiraoka, N Okazaki - Journal of Natural Language …, 2022 - jstage.jst.go.jp
In this study, we propose a method designed to extract named entities and relations from
unstructured text based on table representations. To extract named entities, the proposed …

Addressing NER Annotation Noises with Uncertainty-Guided Tree-Structured CRFs

J Liu, W Liu, Y Chen, J Xu, Z Zhao - Proceedings of the 2023 …, 2023 - aclanthology.org
Real-world named entity recognition (NER) datasets are notorious for their noisy nature,
attributed to annotation errors, inconsistencies, and subjective interpretations. Such noises …

Video question answering supported by a multi-task learning objective

A Falcon, G Serra, O Lanz - Multimedia Tools and Applications, 2023 - Springer
Abstract Video Question Answering (VideoQA) concerns the realization of models able to
analyze a video, and produce a meaningful answer to visual content-related questions. To …

Entity extraction in low resource domains with selective pre-training of large language models

A Mahapatra, SR Nangi, A Garimella… - Proceedings of the …, 2022 - aclanthology.org
Transformer-based language models trained on large natural language corpora have been
very useful in downstream entity extraction tasks. However, they often result in poor …

Genealogical Data-Driven Visits of Historical Cemeteries

A Lo Duca, M Abrate, A Marchetti, M Moretti - Informatics, 2024 - mdpi.com
This paper describes the Integration of Archives and Cultural Places (IaCuP) project, which
aims to integrate information about a historical cemetery, including its map and grave …