[PDF][PDF] Detection of gaming in automated scoring of essays with the IEA

KE Lochbaum, M Rosenstein, P Foltz… - 75th Annual meeting of …, 2013 - Citeseer
KE Lochbaum, M Rosenstein, P Foltz, MA Derr
75th Annual meeting of NCME, 2013Citeseer
In addition to the standard test security issues, automated scoring provides new
opportunities for students to deliberately misrepresent their ability. Gaming of essays can
take many forms including repetition of words and sentences, incorporation of context
irrelevant words, phrases, or sentences, plagiarism, and the insertion of “malicious”
sequences of characters such as HTML web page markup language, which may be aimed
at causing scoring failures. It can also involve a series of sophisticated words arranged in …
Abstract
In addition to the standard test security issues, automated scoring provides new opportunities for students to deliberately misrepresent their ability. Gaming of essays can take many forms including repetition of words and sentences, incorporation of context irrelevant words, phrases, or sentences, plagiarism, and the insertion of “malicious” sequences of characters such as HTML web page markup language, which may be aimed at causing scoring failures. It can also involve a series of sophisticated words arranged in nonsensical ways that confuse a statistical language model into valuing the writing as sophisticated instead of as gibberish. Computer-based approaches to detecting gaming have advantages in that they can sometimes detect subtle statistical patterns in language and plagiarism, which are imperceptible to humans. However, computers may be less sensitive than human scorers to other aspects of writing, such as certain grammar patterns and language features. This talk will describe a general framework used to detect gaming within essays by the Intelligent Essay Assessor™. The development of features, by analyzing aspects of the topic content elaboration, language structure, coherence, and length and their use in detecting gaming, will be described. The talk will describe some of the tradeoffs between full transparency of scoring and detection methods versus obscuring some level of specificity in the algorithms to impede gaming.
Citeseer
以上显示的是最相近的搜索结果。 查看全部搜索结果