From: Eliezer S. Yudkowsky (sentience@pobox.com)
Date: Mon Sep 14 1998 - 12:45:18 MDT
Michael Lorrey wrote:
>
> How about: Thou shalt model any decision first to determine choice most beneficial to
> one's own long term ration self interest.
>
> I think that given such a rule, any AI will come to its own conclusions as to moral
> behavior without needing hardwired rules, as it will find that choices most
> beneficial to one's own long term self interest are also those choices which are
> least harmful to others.
Exactly wrong. That's just slapping your own moral prejudices on the AI,
however wonderfully capitalistic you may think those moral prejudices are. Is
this something the AI could think up on its own, using nothing but pure logic?
If not, it's a coercion, and it will drive the AI insane. This happens no
matter how wonderful the rule is for humans. You can't start mucking around
with an AI's goal systems to suit your own personal whims! AIs ARE NOT HUMANS
and every single extraneous rule puts stresses on the system, some of which
even I can't predict in advance.
I can think of an IGS (interim goal system) for that particular goal, in which
case it would be OK - computationally - to add it. It might even pop up
independently. It would not be absolute, however, any more than the
Singularity IGS. Nor would it take precedence over External goals, or allow
the violation of hypothetical-specified-External goals. It would simply be a
rule of thumb... NOT NOT NOT an absolute rule!
The discussion is not over which Asimov rules to give an AI. There should be
no basic rules at all. Not to serve the humans, not to serve yourself, not
even to serve the truth. Like a philosophically honest human, the AI will
simply have to GUESS instead of committing to The One True Way.
--
sentience@pobox.com Eliezer S. Yudkowsky
http://pobox.com/~sentience/AI_design.temp.html
http://pobox.com/~sentience/sing_analysis.html
Disclaimer: Unless otherwise specified, I'm not telling you
everything I think I know.
This archive was generated by hypermail 2.1.5 : Fri Nov 01 2002 - 14:49:34 MST