From: Robin Lee Powell (rlpowell@digitalkingdom.org)
Date: Sun Oct 10 2010 - 21:13:32 MDT
On Sun, Oct 10, 2010 at 12:07:22AM -0700, Samantha Atkins wrote:
>
> On Oct 7, 2010, at 9:58 AM, John K Clark wrote:
>
> >
> > On Tue, 5 Oct 2010 "Mindaugas Indriunas" <inyuki@gmail.com>
> > said:
> >
> >> It might be that one of the best ways to bring about the
> >> friendly AI is by trying to be very rational about one's own
> >> actions, defining one's own goal of life, and doing it in such
> >> a way that the resulting goal would be the objective good
> >
> >
> > In other words, you're going to try to convince the AI that
> > "objective good" means being good to a human being not to an AI
> > like itself, even though the AI is objectively superior to the
> > human by any measure you care to name. Pushing the virtues of
> > such a slave mentality (sorry, I believe the politically correct
> > term is friendly) to a being much smarter than you are is going
> > to be a very hard sell.
> >
>
> It seems you are starting with an AGI that already has a "mind of
> its own" or its own goal structure. Where did those come from?
John seems to think that AI will magically acquire innate resistance
to whatever goal structure humans give it. We (this list) have been
around this bush many times; you might want to avoid getting sucked
in.
-Robin
-- http://intelligence.org/ : Our last, best hope for a fantastic future. Lojban (http://www.lojban.org/): The language in which "this parrot is dead" is "ti poi spitaki cu morsi", but "this sentence is false" is "na nei". My personal page: http://www.digitalkingdom.org/rlp/
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:01:05 MDT