We call for schools, colleges, and educational assessment programs to stop using computer scoring of student essays written during high-stakes tests.
Every year hundreds of thousands of students write essays for large-scale standardized tests. The scores are used in life-changing decisions. Students are accepted into, placed within, and rejected from educational programs. Graduates are hired or not hired. Teachers are qualified, evaluated, promoted, and fired. Learning institutions are compared, accredited, and punished. Yet in a major disservice to all involved, more and more of these essays are scored not by human readers but by machines.
Let’s face the realities of automatic essay scoring. Computers cannot “read.” They cannot measure the essentials of effective written communication: accuracy, reasoning, adequacy of evidence, good sense, ethical stance, convincing argument, meaningful organization, clarity, and veracity, among others. Independent and industry studies show that by its nature computerized essay rating is
- trivial, rating essays only on surface features such as word size, topic vocabulary, and essay length
- reductive, handling extended prose written only at a grade-school level
- inaccurate, missing much error in student writing and finding much error where it does not exist
- undiagnostic, correlating hardly at all with subsequent writing performance
- unfair, discriminating against minority groups and second-language writers
- secretive, with testing companies blocking independent research into their products
The basic premise of the petition is good: computers can’t score those aspects of writing that people actually care about, and so should not be used for scoring any essays that matter. Of course, some of the alternatives are just about as bad (like the MOOCs that use peer grading by incompetent “peers”), but I signed the petition anyway.
The bottom line is that assessment of writing is difficult, and so is expensive. If you can’t afford to pay for proper evaluation by well-trained human readers, then you can’t afford to use essays as part of an assessment. There are no shortcuts.