Managing what we can measure: Quantifying the susceptibility of automated scoring systems to gaming behavior

D Higgins, M Heilman - Educational Measurement: Issues and …, 2014 - Wiley Online Library
As methods for automated scoring of constructed‐response items become more widely
adopted in state assessments, and are used in more consequential operational …

Using automated feedback to improve writing quality: Opportunities and challenges

J Wilson, GN Andrada - Handbook of research on technology tools …, 2016 - igi-global.com
Writing skills are essential for success in K-12 and post-secondary settings. Yet, more than
two-thirds of students in the United States fail to achieve grade-level proficiency in writing …

Human scoring versus automated scoring for english learners in a statewide evidence-based writing assessment

Y Vo, H Rickels, C Welch, S Dunbar - Assessing Writing, 2023 - Elsevier
This study examined the validity evidence of automated scores across English learners
(ELs) and non-EL test takers in a statewide summative writing assessment. Writing …

[PDF][PDF] Psychometric considerations when using deep learning for automated scoring

S Lottridge, C Ormerod, A Jafari - Advancing natural language …, 2023 - library.oapen.org
Automated scoring refers to the use of statistical and computational linguistic methods to
assign scores or labels to examinee responses to unconstrained open-ended test items …

Combining human and automated scores for the improved assessment of non-native speech

SY Yoon, K Zechner - Speech Communication, 2017 - Elsevier
In this study, we propose an efficient way to combine human and automated scoring to
increase the reliability and validity of a system used to assess spoken responses in the …

Essay Quality Signals as Weak Supervision for Source-Based Essay Scoring.

H Zhang, D Litman - Grantee Submission, 2021 - ERIC
Human essay grading is a laborious task that can consume much time and effort. Automated
Essay Scoring (AES) has thus been proposed as a fast and effective solution to the problem …

Atypical inputs in educational applications

SY Yoon, A Cahill, A Loukina, K Zechner… - Proceedings of the …, 2018 - aclanthology.org
In large-scale educational assessments, the use of automated scoring has recently become
quite common. While the majority of student responses can be processed and scored …

[PDF][PDF] Comparing the robustness of deep learning and classical automated scoring approaches to gaming strategies

S Lottridge, B Godek, A Jafari, M Patel - 2021 - files.portal.cambiumast.com
The state of the art in machine-learning scoring has evolved in recent years to achieve gains
in accuracy in a number of predictive tasks. Older models used feature-based approaches …

[PDF][PDF] Developing an e-rater Advisory to Detect Babel-generated Essays

A Cahill, M Chodorow, M Flor - Journal of Writing Analytics, 2018 - academia.edu
• Background: It is important for developers of automated scoring systems to ensure that their
systems are as fair and valid as possible. This commitment means evaluating the …

[PDF][PDF] Automatic Detection of Off-Topic Spoken Responses Using Very Deep Convolutional Neural Networks.

X Wang, SY Yoon, K Evanini, K Zechner, Y Qian - INTERSPEECH, 2019 - isca-archive.org
Test takers in high-stakes speaking assessments may try to inflate their scores by providing
a response to a question that they are more familiar with instead of the question presented in …