Computer scoring and quality of thought in assessing writing

This chapter describes the use of computers to score writing tests, canvasses the strengths and weaknesses of computer scoring, and examines what the use of computer scoring shows about the construct of writing ability. The chapter reviews, from a distance, some aspects of the development since 1998 of the e-rater computer scoring software of the Educational Testing Service. Some of the published attempts to explore the validity of e-rater scoring are reviewed, with particular emphasis on the attempt to ground the use of computer scoring on a clear view of writing ability that could give a substantive rationale for the use of such software, and the interpretation of the scores it produces. Arising from this discussion, the different ways in which writing ability can be understood are examined, with particular reference to the content/thought versus form/language dualism. The attempts to have computer software deal with the crucial issue of the quality of thought in writing are discussed. The changes from the first to the second version of e-rater are reviewed. The development of a nuanced and sophisticated view of writing ability, and the possible use of such a view as the basis for a specific and circumscribed use of computer scoring in the future, is considered.