The human unlikeness of neural language models in next-word prediction
CL Jacobs, AD McCarthy - Proceedings of the Fourth Widening …, 2020 - aclanthology.org
The training objective of unidirectional language models (LMs) is similar to a
psycholinguistic benchmark known as the cloze task, which measures next-word
predictability. However, LMs lack the rich set of experiences that people do, and humans
can be highly creative. To assess human parity in these models' training objective, we
compare the predictions of three neural language models to those of human participants in a
freely available behavioral dataset (Luke & Christianson, 2016). Our results show that while …
psycholinguistic benchmark known as the cloze task, which measures next-word
predictability. However, LMs lack the rich set of experiences that people do, and humans
can be highly creative. To assess human parity in these models' training objective, we
compare the predictions of three neural language models to those of human participants in a
freely available behavioral dataset (Luke & Christianson, 2016). Our results show that while …