From: Damien Broderick (damien@ariel.ucs.unimelb.edu.au)
Date: Sun Sep 13 1998 - 03:19:02 MDT
At 09:06 AM 9/11/98 -0500, Eliezer wrote:
>SF, yes. I have, for example, read "Tik-Tok" and "Roderick"
When do you get time to sleep? But I'm glad you've read them; Sladek, as
you note, is very funny, and astute and subversive. I remembered
(misremembered?) his playing with the strictly ineradicable mutability of
interpretation as a key to robot disobedience.
>> < Never allow arbitrary, illogical, or untruthful goals to enter the AI. >
>> reflects a touching faith in human powers of understanding and consistency.
>I'm not quite sure what you mean by this.
Isn't it obvious? How can any limited mortal know in advance what another
intelligence, or itself at a different time and in other circumstances,
might regard as `arbitrary, illogical, or untruthful'? Popper spank.
>I quote from Asimov's "The Complete Robot": <snip I. A. AI boast >
Okay, of course a man would feel obliged to do a bit of swaggering if he'd
carelessly invented the Three Laws instead of the telecommunications
satellite. But I still don't think Asimov took the notion all that
seriously. In ROBOT VISIONS (which I grab from the shelf at random... at
random... at random... I read:
`Of course, these laws are expressed in words, which is an imperfection.
In the positronic brain, they are competing positronic potentials that are
best expressed in terms of advanced mathematics... However, even so, there
are clear ambiguities. [Instances are cited: scary orders given to robots
by ignorant children, the deranged, Jonathan Grimes, that sort of thing.]
These ambiguities are not shortcomings as far as a writer is concerned. If
the Three Laws were perfect and unambiguous there would be no room for
stories... [Eventually] a sufficiently advanced robot might find it
necessary to consider the prevention of harm to humanity generally as
taking precedence over the prevention of harm to an individual. This is
called the "Zeroth Law...".'
And so on. This is a simple case of one discourse reframing the content of
another, capable of turning any injunction 180 degrees - which is, as Eli
notes, what Greg Egan does so entertainingly in QUARANTINE. Unstable
definitions? Sure, a sub-set.
>> More seriously, the topic of discursive reframing (which makes a total hash
>> of any `Prime Directive' imaginable)
>I'm not sure what
>you mean by it making a hash of the "Prime Directive"; I suspect you meant
>"Three Laws".
Those too, of course, but look again at the wording:
< arbitrary, illogical, or untruthful >
By the standard of what consensus community? What practice of discourse
and action in the world? It is illogical and dangerous to walk toward the
horizon, because eventually you will fall off the edge of the world.
Re-frame: the world is spherical. Oh, okay. When is an act a clear-cut
instance of `sexual relations'? Sometimes, as Freud insisted, a cigar is
just a cigar. Sometimes it's an impeachment.
Damien Broderick
This archive was generated by hypermail 2.1.5 : Fri Nov 01 2002 - 14:49:34 MST