Machine “Readers”?



Those of us who teach composition know the difficulty of convincing students to think of audience as they write: Just who are they addressing? What do they expect in response? Why are they saying something? Writing is about convincing, entertaining, conveying, demanding, contacting….

However you describe it, writing is as much a two-way street as talking. Even when we talk/write to ourselves, we play two roles… two human roles.

There’s a new petition against machine scoring of writing. Please sign it, if you can… especially if you teach or require writing in any form (and most of us do):

Every year hundreds of thousands of students write essays for large-scale standardized tests. The scores are used in life-changing decisions. Students are accepted into, placed within, and rejected from educational programs. Graduates are hired or not hired. Teachers are qualified, evaluated, promoted, and fired. Learning institutions are compared, accredited, and punished. Yet in a major disservice to all involved, more and more of these essays are scored not by human readers but by machines.

Let’s face the realities of automatic essay scoring. Computers cannot “read.” They cannot measure the essentials of effective written communication: accuracy, reasoning, adequacy of evidence, good sense, ethical stance, convincing argument, meaningful organization, clarity, and veracity, among others. Independent and industry studies show that by its nature computerized essay rating is

  • trivial, rating essays only on surface features such as word size, topic vocabulary, and essay length
  • reductive, handling extended prose written only at a grade-school level
  • inaccurate, missing much error in student writing and finding much error where it does not exist
  • undiagnostic, correlating hardly at all with subsequent writing performance
  • unfair, discriminating against minority groups and second-language writers
  • secretive, with testing companies blocking independent research into their products….
  • In sum, current machine scoring of essays is not defensible, even when procedures pair human and computer raters. It should not be used in any decision affecting a person’s life or livelihood and should be discontinued for all large-scale assessment purposes.

    If you agree, please follow the link above and sign the petition.

    Update: Thanking Jonathan Rees for his piece drawing it to my attention, I urge reading Mark Cheathem’s post “Selling the Matrix Revolution in Higher Education.” I’m feeling that we are being treated like Bob Dylan’s Mr. Jones: “There’s something happening here but you don’t know what it is.” Thing is, I think we actually do… but we need to start acting like it or we’re going to be herded into either a corral or a stampede… I am not sure which… by the forces of profit through machine.