From: James Higgins (jameshiggins@earthlink.net)
Date: Thu Apr 05 2001 - 22:19:30 MDT
At 01:39 PM 4/5/2001 -0400, Brian Atkins wrote:
>Samantha Atkins wrote:
You two related or something?
> > I don't see how this follows. If you upload a human who quickly
> > self-improves ver capabilities and becomes an SI and if
> > super-intelligence brings with it expanded moral/ethical understanding
> > then I see no reason this combination is less trustworthy than starting
> > from scratch and only putting in what you believe should be there in the
> > beginning. Yes a lot of evolved complicated behavior and conditioning
> > is not present in the AI. But some of that complicated behavior and
> > conditioning is also the bed of universal compassion and utter
> > Friendliness.
> >
>
>Lot of ifs there...
Well, everything we talk about would need to be qualified a hundred
different ways if we got down to the nitty gritty. At this point, this is
more SF than a science. That will change, however, over the next few years.
>What it seems to come down to is you are either relying on objective
>morality (in which an AI should do better since it has less evolved
>crap to deal with), or a natural convergence to the "Friendly zone". In
>which case we also argue that a properly designed AI should easily
>outperform a human attempting to upgrade him/herself. The reason I think
>is easy to see: you can't really predict in advance which particular
>human will become utterly Friendly vs. which particular human will become
>the next Hitler when presented with the total power uploading/becoming a
>SI would give them. History has shown a tendency for power to corrupt
>humans. At least with an AI we can sharply reduce the risks by a) designing
>it right b) testing testing testing
BS. Testing means jack when you are talking about something that will
complete reinvent itself a thousand times. And I have a feeling that the
design will have little long-term impact also. Certainly, the design will
influence how well the AI starts off, and how well it upgrades itself the
first few times or so. But beyond the first dozen or less redesigns, it
will have little, if any, resemblance to the original. So excuse me if I
don't have much faith.
>You want to talk about who designs the first AI? Well who decides who gets
>to be the first upload?
Hey, that is easy. I'll be first. ;)
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:36 MDT