From: Smigrodzki, Rafal (SmigrodzkiR@msx.upmc.edu)
Date: Thu Mar 21 2002 - 15:36:36 MST
Eliezer S. Yudkowsky [mailto:sentience@pobox.com]
Wrote:
Someone wants to shoot me. Is it moral for me to act
counter to his
explicit wishes and dodge? Sure. Is it moral for a FAI to
intersect the
bullet because my wishes about my life take precedence over
his wishes about
my life? Sure.
### How do you know? Do you presume to judge a god, mortal
one? Maybe it's in your best interest to bite the bullet. Deity-level
intelligence could surprise you.
----------
> Also, what if the FAI arrives at the
conclusion that the
> principle of autonomy as applied to sentients above 85 IQ
points, plus
> majority rule trump all other principles? If so, then it
might act to
> fulfill the wishes of the majority, even if it means
destruction of some
> nice folks.
Why would an FAI conclude this?
### See above.
-------
> ### I have no doubt the SAI will be quite
impressive but
> without being able to follow its reasoning steps you will
be unable to
> detect FoF
It's an SIs responsibility to do that kind of maintenance.
Once there's an
SI around, whether you detect FoF is quite pointless...
### Point taken. That's why I want to upload and enhance
myself ASAP.
------
> ### OK. I understand. But once an
adaptation is accepted by
> a declarative reasoning process into a person's goal
system, it can become a
> subgoal. So altruism is both an adaptation *and* a subgoal
in some persons.
Yes, but they are conceptually separate. I make use of my
altruistic
hardware but I apply reasoned corrective factors where I
expect hardware
altruism to differ from normative altruism.
### This is an interesting thought. If Geraint Rees is
reading this post, he might perhaps be able to point us to fMRI studies on
the localization of ethical (altruism as adaptation) versus meta-ethical
(altruism as subgoal) reasoning. But the referent (idea of altruism) stays
the same, no matter which part of the cortex is implementing it.
Rafal
This archive was generated by hypermail 2.1.5 : Sat Nov 02 2002 - 09:13:03 MST