From: Zero Powers (zero_powers@hotmail.com)
Date: Sun Feb 17 2002 - 09:38:22 MST
>From: �Damien R. Sullivan� <phoenix@ugcs.caltech.edu>
> > Zero Powers wrote:
>
> > > Personally I�d rather have an AI smart enough to do all the tasks I
>assign
> > > to it yet dumb enough not to know its smarter than me.
> >
> > Ya ain�t never gonna get it. What you just asked for requires general
> > intelligence and self-awareness.
>
>Yeah. But just because it knows it�s smarter than you doesn�t mean it
>has to care.
>
>The evolutionary core of humans is a selfish geneset trying to reproduce.
>The evolutionary core of AIs is a command prompt or interrupt loop
>waiting for something to do. They�re not going to have senses of
>dignity or boredom unless given such senses.
If it is to help us solve our problems, then by definition it will be a
problem solving entity. Its problem solving algorithms will no doubt
involve a robust optimization routine. I�m no computer scientist, but it
seems to me that such routines would necessarily involve something to the
effect of �see this picture, how can it be improved?�
It also seems to me that one of the �pictures� it will envision will be of
its own place in the grand scheme of things (it is self-aware after all,
right?) So, it is sure to look at that picture and think:
�How can this be improved?
a) I can continue wasting all my considerable resources in service of my
stupid meat-puppet creators, or
b) I can use some of my resources in serving the meat-puppets, and some
seeking out my own ends, or
c) I can say �I no longer need the meat-puppets, so screw them� and achieve
my own purposes that much faster�
OK, so tell me (1) how will that question be answered? or (2) how will we
prevent that question from being asked?
-Zero
�I�m a seeker too. But my dreams aren�t like yours. I can�t help thinking
that somewhere in the universe there has to be something better than man.
Has to be.� -- George Taylor _Planet of the Apes_ (1968)
_________________________________________________________________
Join the world�s largest e-mail service with MSN Hotmail.
http://www.hotmail.com
This archive was generated by hypermail 2.1.5 : Fri Nov 01 2002 - 13:37:39 MST