From: Eliezer S. Yudkowsky (sentience@pobox.com)
Date: Tue Sep 14 1999 - 20:00:15 MDT
den Otter wrote:
>
> Responsible and ethical people would probably use Asimov's
> robotics laws to control the AI, which may or may not work
> (probably not). How the AI evolves may very well be "fundamentally"
> beyond our control. So...I'd join the *uploading* team if you
> have serious future plans. Load me up, Scotty!
Dammit, Otter, if an entity that started out as uploaded Otter managed
to keep *any* of your motivations through Transcendence, selfish or
otherwise, you could use the same pattern to create a reliably
benevolent Power. I mean, let's look at the logic here:
OTTER and ELIEZER, speaking in unison: "The mind is simply the result
of evolution, and our actions are the causal result of evolution. All
emotions exist only as adaptations to a hunter-gatherer environment, and
thus, to any Power, are fundamentally disposable."
ELIEZER: "If there's one set of behaviors that isn't arbitrary, it's
logic. When we say that two and two make four, it's copied from the
laws of physics and whatever created the laws of physics, which could
turn out to be meaningful. All the other stuff is just survival and
reproduction; we know how that works and it isn't very interesting."
OTTER: "All the emotions are arbitrary evolved adaptations, except
selfishness, which alone is meaningful."
This just says "Thud". To use a Hofstadterian analogy, it's like:
abc->abd::xyz->?
Hofstadter presents a number of answers, including:
xya
xyd
wyz (the best)
and one, which Hofstadter says "definitely presents a very loud 'Thud!'",
dyz
Otter's answer makes a similar 'Thud!' to my ears. He gets as far as
seeing the essential arbitrariness of it all, but instead of trying to
pick out something that isn't arbitrary, he picks out selfishness and
canonizes it. dyz.
--
sentience@pobox.com Eliezer S. Yudkowsky
http://pobox.com/~sentience/tmol-faq/meaningoflife.html
Running on BeOS Typing in Dvorak Programming with Patterns
Voting for Libertarians Heading for Singularity There Is A Better Way
This archive was generated by hypermail 2.1.5 : Fri Nov 01 2002 - 15:05:09 MST