From: Eliezer S. Yudkowsky (sentience@pobox.com)
Date: Tue May 21 2002 - 13:14:03 MDT
Stephen Reed wrote:
>
> On Tue, 21 May 2002, Eliezer S. Yudkowsky wrote:
>
> > If you stripped away all the English names from all AI content and replaced
> > them with randomly generated strings, so that the AI could no longer handle
> > human-supplied problems and human-supplied data which required
> > identification or invocation of concepts by English name, what capabilities
> > would the AI still possess?
>
> Could you clarify a bit in light of how Cyc handles term names? In Cyc,
> to allow easy renames, the term name is not the identity of the term, it
> is merely an attribute. The Cyc term identity is a GUID (128-bit
> universally unique id) generated by the operating system upon term
> creation.
In this case, the conceptual experiment I describe above would leave the
GUID constant and replacing the term name with a randomly generated string.
Scrambling the GUIDs is not something that any mind could be expected to
tolerate.
> We have considered an internal test, what we call the GENSYM
> test, that follows your outline:
>
> Take a term, replace its name with a GENSYM (sequentially numbered
> meaningless string), and with its comment hidden, try to identify the
> concept. We would also hide the English words that lexify the term.
If you perform this test on only one term at a time, so that you're trying
to identify an unnamed concept by its relation to other English terms, then
this may uncover interesting ambiguities in the description language. If
Cyc can do it successfully, I think that would show that a Cyc's-eye-view is
sufficient to say something important about how the strings that humans use
are related to each other, but it doesn't necessarily show, in itself, that
a Cyc's-eye-view includes the meaning behind the strings. Of course, some
GOFAI philosophers are famous for insisting that the relations between words
constitute the whole of any one word's meaning, but as you know, I think
that the perceptual patterns invoked by words also have something to do with
it.
> If we went beyond this test in Cyc, and replaced all the concept names
> with random strings, but kept the lexical information, then Cyc would
> operate fine. In that the Knowledge Entry and Query tools based on the
> lexicon would still function. But if the lexical information were
> removed, then you would be unable to easily work with Cyc, but would have
> to reconstruct a mental model of the concept hierarcy from the top down.
The question is whether the concept hierarchy would be reconstructable at
all. I think that if an AI has perceptual concepts, then you would be able
to rapidly identify individual words in those modalities that the AI
possessed through the usual method of showing the AI a picture, or running
the AI through an experience, and then seeing what concept the AI used to
label it. I also think that being able to reconstruct a scrambled concept
net, in this way, is a necessary condition for saying that the AI possesses
knowledge about the concept behind the word, and not just knowledge about
how the word is likely to be used in proximity to other words.
It's been a few years now since I read up on Cyc and CycL, but I remember
thinking at the time that the reflective portions of the ontology, and
perhaps things like "verb", "follows temporally" and "five", would probably
be the portions of the concept network that could be reconstructed following
a global scramble. I think that from a Cyc's-eye-view, Cyc may know
something about the grammatical structure of English strings; it knows which
English strings are synonyms for English strings, and which English strings
are properties of English strings that describe categories, and which
English strings denote events that cause events denoted by other English
strings to occur, and so on, but it doesn't have any idea which events and
categories the English strings correspond to.
Thus, I think that for AI in general and Cyc in particular, the global
scramble thought experiment does say something critical about which concepts
are anchored independently of their term names. I will confess to being
something of a Cyc skeptic because I think that concepts which lack such
independent anchoring are not real knowledge. If I had to pick an
incremental direction for Cyc (as opposed to a complete philosophical
revision), I think it would be focusing on those Cyc concepts that can be
grounded in the complex data of Cyc's internals - i.e., teaching Cyc
perceptual concepts for its own internals.
-- -- -- -- --
Eliezer S. Yudkowsky http://intelligence.org/
Research Fellow, Singularity Institute for Artificial Intelligence
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:39 MDT