Linguistic microfeatures to predict L2 writing proficiency: A case study in automated writing evaluation

SA Crossley, K Kyle, LK Allen, L Guo… - Journal of Writing …, 2014 - escholarship.org
Journal of Writing Assessment, 2014escholarship.org
This study investigates the potential for linguistic microfeatures related to length, complexity,
cohesion, relevance, topic, and rhetorical style to predict L2 writing proficiency.
Computational indices were calculated by two automated text analysis tools (Coh-Metrix and
the Writing Assessment Tool) and used to predict human essay ratings in a corpus of 480
independent essays written for the TOEFL. A stepwise regression analysis indicated that six
linguistic microfeatures explained 60% of the variance in human scores for essays in a test …
This study investigates the potential for linguistic microfeatures related to length, complexity, cohesion, relevance, topic, and rhetorical style to predict L2 writing proficiency. Computational indices were calculated by two automated text analysis tools (Coh-Metrix and the Writing Assessment Tool) and used to predict human essay ratings in a corpus of 480 independent essays written for the TOEFL. A stepwise regression analysis indicated that six linguistic microfeatures explained 60% of the variance in human scores for essays in a test set, providing an exact accuracy of 55% and an adjacent accuracy of 96%. To examine the limitations of the model, a post-hoc analysis was conducted to investigate differences in the scoring outcomes produced by the model and the human raters for essays with score differences of two or greater (N = 20). Essays scored as high by the regression model and low by human raters contained more word types and perfect tense forms compared to essays scored high by humans and low by the regression model. Essays scored high by humans but low by the regression model had greater coherence, syntactic variety, syntactic accuracy, word choices, idiomaticity, vocabulary range, and spelling accuracy as compared to essays scored high by the model but low by humans. Overall, findings from this study provide important information about how linguistic microfeatures can predict L2 essay quality for TOEFL-type exams and about the strengths and weaknesses of automatic essay scoring models.
escholarship.org
以上显示的是最相近的搜索结果。 查看全部搜索结果

Google学术搜索按钮

example.edu/paper.pdf
搜索
获取 PDF 文件
引用
References