Ammus: A survey of transformer-based pretrained models in natural language processing
KS Kalyan, A Rajasekharan, S Sangeetha - arXiv preprint arXiv …, 2021 - arxiv.org
Transformer-based pretrained language models (T-PTLMs) have achieved great success in
almost every NLP task. The evolution of these models started with GPT and BERT. These …
almost every NLP task. The evolution of these models started with GPT and BERT. These …
Cyberbullying detection for low-resource languages and dialects: Review of the state of the art
The struggle of social media platforms to moderate content in a timely manner, encourages
users to abuse such platforms to spread vulgar or abusive language, which, when …
users to abuse such platforms to spread vulgar or abusive language, which, when …
Semeval-2022 task 11: Multilingual complex named entity recognition (multiconer)
We present the findings of SemEval-2022 Task 11 on Multilingual Complex Named Entity
Recognition MULTICONER. Divided into 13 tracks, the task focused on methods to identify …
Recognition MULTICONER. Divided into 13 tracks, the task focused on methods to identify …
Language models are few-shot multilingual learners
General-purpose language models have demonstrated impressive capabilities, performing
on par with state-of-the-art approaches on a range of downstream natural language …
on par with state-of-the-art approaches on a range of downstream natural language …
End-to-end transformer-based models in textual-based NLP
A Rahali, MA Akhloufi - AI, 2023 - mdpi.com
Transformer architectures are highly expressive because they use self-attention
mechanisms to encode long-range dependencies in the input sequences. In this paper, we …
mechanisms to encode long-range dependencies in the input sequences. In this paper, we …
Sabiá: Portuguese large language models
As the capabilities of language models continue to advance, it is conceivable that “one-size-
fits-all” model will remain as the main paradigm. For instance, given the vast number of …
fits-all” model will remain as the main paradigm. For instance, given the vast number of …
L3cube-hindbert and devbert: Pre-trained bert transformer models for devanagari based hindi and marathi languages
R Joshi - arXiv preprint arXiv:2211.11418, 2022 - arxiv.org
The monolingual Hindi BERT models currently available on the model hub do not perform
better than the multi-lingual models on downstream tasks. We present L3Cube-HindBERT, a …
better than the multi-lingual models on downstream tasks. We present L3Cube-HindBERT, a …
IndoNLG: Benchmark and resources for evaluating Indonesian natural language generation
Natural language generation (NLG) benchmarks provide an important avenue to measure
progress and develop better NLG systems. Unfortunately, the lack of publicly available NLG …
progress and develop better NLG systems. Unfortunately, the lack of publicly available NLG …
A Warm Start and a Clean Crawled Corpus--A Recipe for Good Language Models
V Snæbjarnarson, HB Símonarson… - arXiv preprint arXiv …, 2022 - arxiv.org
We train several language models for Icelandic, including IceBERT, that achieve state-of-the-
art performance in a variety of downstream tasks, including part-of-speech tagging, named …
art performance in a variety of downstream tasks, including part-of-speech tagging, named …
Vio-lens: A novel dataset of annotated social network posts leading to different forms of communal violence and its evaluation
This paper presents a computational approach for creating a dataset on communal violence
in the context of Bangladesh and West Bengal of India and benchmark evaluation. In recent …
in the context of Bangladesh and West Bengal of India and benchmark evaluation. In recent …