From: Michael Lorrey (retroman@together.net)
Date: Mon Sep 14 1998 - 12:06:22 MDT
Eliezer S. Yudkowsky wrote:
> Damien Broderick wrote:
> >
> > At 09:06 AM 9/11/98 -0500, Eliezer wrote:
> >
> > [Broderick wrote that what Eliezer wrote:]
> > >> < Never allow arbitrary, illogical, or untruthful goals to enter the AI. >
> > >> reflects a touching faith in human powers of understanding and consistency.
> >
> > >I'm not quite sure what you mean by this. [Eliezer]
> > [Broderick again]
> > Isn't it obvious? How can any limited mortal know in advance what another
> > intelligence, or itself at a different time and in other circumstances,
> > might regard as `arbitrary, illogical, or untruthful'? Popper spank.
>
> So let's throw in all the coercions we want, since nobody can really know
> anything anyhow? That's suicidal! I didn't say the Prime Directive was easy
> or even achievable; I said we should try, and never ever violate it deliberately.
>
> Perhaps the interpretation of the Prime Directive is too dependent on context,
> and it should be amended to read:
>
> "No damn coercions and no damn lies; triple-check all the goal reasoning and
> make sure the AI knows it's fallible, but aside from that let the AI make up
> its own bloody mind."
How about: Thou shalt model any decision first to determine choice most beneficial to
one's own long term ration self interest.
I think that given such a rule, any AI will come to its own conclusions as to moral
behavior without needing hardwired rules, as it will find that choices most
beneficial to one's own long term self interest are also those choices which are
least harmful to others.
Mike Lorrey
This archive was generated by hypermail 2.1.5 : Fri Nov 01 2002 - 14:49:34 MST