Re: Why would AI want to be friendly?

From: Eugene Leitl (eugene.leitl@lrz.uni-muenchen.de)
Date: Wed Sep 27 2000 - 02:10:47 MDT


Samantha Atkins writes:

> Or you could start with humans and continuously augment (voluntarily)
> with first external but more and more integrated and then internal
> hardware and software. This seems to me the best way to keep humans in

This method is

* slow

* technically demanding (biocompatibility; probably requires nanotechnology)

* has high ethical threshold and is risky (neurosurgery is no peanuts)

* attempts to integrate two very different paradigms: neuro and
  digital, which requires a very good understanding of the wet substrate

It would work in principle, provided in the meantime no AI grown by
evolutionary algorithms emerges. (This is unlikely, because in 20-30
years we should have molecular circuitry, and hence enough computing
performance to breed an AI from scratch, while augmentation will have
made scarcely any headway in that time frame). Because of explosive
kinetics of the self-enhancing positive autofeedback process of the
AI, the cyborg wannabees would be just as left in the dust as
unaugmented people.

> the loop and to end up with something human compatible and reasonably
> likely to be friendly and caring about humanity.
 
Assuming, the transition will be indeed so slow. Convergent evolution
would seem to require that ALife AI and uploaders would be
undistinguishable. Because uploaders would be probably slower to
converge initially due to evolutionary ballast, this probably means
that they will be blown away by ALife AI, if latter emerges at
relatively early step of the game.

> But what a great resources to hook into the WebMind or to have at the
> disposal of more capable AI! Don't throw that work out. It is a useful
> piece. If nothing else it is a huge glob of training material.

A search engine with Cyc functionality is indeed very useful, but only
as long as we can't create real AI. It *is* a useful piece, as it
demonstrates another failure of a given approach.



This archive was generated by hypermail 2.1.5 : Fri Nov 01 2002 - 15:31:14 MST