From: hal@finney.org
Date: Fri Jul 16 1999 - 09:58:23 MDT
Jeff Davis, <jdavis@socketscience.com>, writes:
> NEW WORD ORDER
>
> THE ATTACK OF THE INCREDIBLE GRADING
> MACHINE
>
> BY CLIVE THOMPSON
>
> At or near:
>
> http://www.linguafranca.com/
Actually at http://www.linguafranca.com/9907/nwo.html
This is a technique for grading student essay papers which is supposedly
as accurate as professors' grades:
: But the technique Landauer came up with is an altogether different
: beast. His software completely ignores style, grammar, and syntax. In
: fact, it relies on none of the familiar rules of language at all. It
: concerns itself solely with the following question: Does the student's
: essay use words appropriate to the subject matter? To answer this
: question, Landauer's software performs a series of operations. First,
: it assembles a customized database of texts on the assigned topic. It
: measures the spatial relationships among all the words in these texts,
: noting where each word appears and which other words it is near. The
: software then performs a similar analysis on the student essay. Finally,
: it compares the student essay with the texts in its database. The theory
: behind the method is this: For any given essay, good content is a function
: of using certain words in the vicinity of certain other words, and that
: accomplishment can be expressed numerically.
:
: It sounds too complicated (or too simple) to be any good, but Landauer
: and his team claim their program can crank out grades as accurately as
: any professor.
This has an obvious flaw. The professor is checking for style, syntax,
and logic. If students know that this program is going to check their
papers, all they need to do is to write something that produces the
right words in roughly the right positions, without regard to sentence
structure.
There have been "travesty generators" around for years which will take
a text and chop it up into words or phrases and output them at random.
The resulting mishmash almost sorta makes sense when you look at it a
piece at a time, but it is actually complete gibberish. The text editor
Emacs has had for 20 years a function called dissociated-press which
does this. My guess is that loading a couple of standard references no
the topic into your buffer and running dissociated-press will produce
something that passes these guys' test with flying colors, but which would
get a big fat F in an instant if a human looked at it.
Hal
This archive was generated by hypermail 2.1.5 : Fri Nov 01 2002 - 15:04:29 MST