Sbert-wk: A sentence embedding method by dissecting bert-based word models
Sentence embedding is an important research topic in natural language processing (NLP)
since it can transfer knowledge to downstream tasks. Meanwhile, a contextualized word …
since it can transfer knowledge to downstream tasks. Meanwhile, a contextualized word …
A closer look at how fine-tuning changes BERT
Y Zhou, V Srikumar - arXiv preprint arXiv:2106.14282, 2021 - arxiv.org
Given the prevalence of pre-trained contextualized representations in today's NLP, there
have been many efforts to understand what information they contain, and why they seem to …
have been many efforts to understand what information they contain, and why they seem to …
Compositionality and Sentence Meaning: Comparing Semantic Parsing and Transformers on a Challenging Sentence Similarity Dataset
One of the major outstanding questions in computational semantics is how humans integrate
the meaning of individual words into a sentence in a way that enables understanding of …
the meaning of individual words into a sentence in a way that enables understanding of …
DirectProbe: Studying representations without classifiers
Y Zhou, V Srikumar - arXiv preprint arXiv:2104.05904, 2021 - arxiv.org
Understanding how linguistic structures are encoded in contextualized embedding could
help explain their impressive performance across NLP@. Existing approaches for probing …
help explain their impressive performance across NLP@. Existing approaches for probing …
[HTML][HTML] Extracting Sentence Embeddings from Pretrained Transformer Models
L Stankevičius, M Lukoševičius - Applied Sciences, 2024 - mdpi.com
Pre-trained transformer models shine in many natural language processing tasks and
therefore are expected to bear the representation of the input sentence or text meaning …
therefore are expected to bear the representation of the input sentence or text meaning …
Morality is Non-Binary: Building a Pluralist Moral Sentence Embedding Space using Contrastive Learning
J Park, E Liscio, PK Murukannaiah - arXiv preprint arXiv:2401.17228, 2024 - arxiv.org
Recent advances in NLP show that language models retain a discernible level of knowledge
in deontological ethics and moral norms. However, existing works often treat morality as …
in deontological ethics and moral norms. However, existing works often treat morality as …
From general language understanding to noisy text comprehension
Obtaining meaning-rich representations of social media inputs, such as Tweets
(unstructured and noisy text), from general-purpose pre-trained language models has …
(unstructured and noisy text), from general-purpose pre-trained language models has …
How to probe sentence embeddings in low-resource languages: On structural design choices for probing task evaluation
Sentence encoders map sentences to real valued vectors for use in downstream
applications. To peek into these representations-eg, to increase interpretability of their …
applications. To peek into these representations-eg, to increase interpretability of their …
Semantic composition in visually grounded language models
R Pandey - arXiv preprint arXiv:2305.16328, 2023 - arxiv.org
What is sentence meaning and its ideal representation? Much of the expressive power of
human language derives from semantic composition, the mind's ability to represent meaning …
human language derives from semantic composition, the mind's ability to represent meaning …
Data-driven models and computational tools for neurolinguistics: a language technology perspective
In this paper, our focus is the connection and influence of language technologies on the
research in neurolinguistics. We present a review of brain imaging-based neurolinguistic …
research in neurolinguistics. We present a review of brain imaging-based neurolinguistic …