From: Robin Hanson (rhanson@gmu.edu)
Date: Thu Jun 21 2001 - 12:27:02 MDT
At 10:06 AM 6/21/2001 -0700, you wrote:
>I've been a little skeptical about CYC since reading this independent
>review, http://boole.stanford.edu/pub/cyc.report, which did not find the
>performance of the system to meet expectations. ...
Immediately after mentioning that report, the article Eugene cited says:
>But in other tests, Cyc blew away the competition as decisively as Eurisko's
>space cruisers. In July 1998, the Pentagon put Cyc and a dozen other AI
>systems through their analytical paces, giving each team a package of 300
>pages of abstruse data to program in their systems and following up with a
>series of complicated strategic queries. Cyc scored better than all the
>other systems put together, according to the company, leading the Pentagon
>to make it the core of a new experimental program aimed at developing large
>knowledge bases.
It would be nice to learn more about this competition. But assuming it
was managed in good faith, this seems strong evidence that Cyc is much
better than the competition. Surely Cyc is not as good as many people
expect it to be, but it may be far better than they have any right to
expect it to be. Cyc is probably still a long way off from getting "most"
of common sense knowledge, but it is also probably much farther along that
the other efforts. It is all well and good for Vaughan Pratt to call for
more objective measures of progress, but if he thinks it so important maybe
he should roll up his sleeves and create such better measures. If Cyc
does eventually succeed, all the other AI researchers should be called
to account for what they were doing instead of helping to improve Cyc.
Robin Hanson rhanson@gmu.edu http://hanson.gmu.edu
Asst. Prof. Economics, George Mason University
MSN 1D3, Carow Hall, Fairfax VA 22030-4444
703-993-2326 FAX: 703-993-2323
This archive was generated by hypermail 2.1.5 : Sat Nov 02 2002 - 08:08:14 MST