From: GBurch1@aol.com
Date: Sun Oct 03 1999 - 11:19:28 MDT
- - - - [ Part Two of a VERY long post ] - - - -
> > Where does this conclusion come from? Simple: The Golden Rule. Minds
are
> > a
> > special class of moral object BECAUSE they are also moral subjects. In
> > other
> > words, we have to treat minds differently from other moral objects
because
> > they are like "us", i.e. there is a logical reflexivity in contemplating
a
> > mind as a moral object.
>
> But military commanders will [unhappily] order their troops into situations
> where they know some of those moral subjects *will* be killed. If I want
> to use my "Brhump" in an experiment in which I know (and it knows) that
> its eventually the cutting room floor for it, it seems justified if
> I can come up with a good reason for it. I'm facing a really difficult
> problem that simply *has* to be solved; 1 brain isn't enough for it;
> create 2 brains have them work on seperate paths; select the brain
> that comes up with the best solution and kill the other brain).
> Thats what nature does and most of us don't don't see nature as
> being "morally" wrong.
First, the ethics of war raise very acute moral issues. People make the
choice to sacrifice themselves for deeply-held values sometimes, it is true.
They subject themselves to a command structure in which one person is given
the authority to make such decisions because of the moral extremity of
warlike situations (or out of simple stupidity). But such matters are not
the same as destroying an independent mind without its consent.
In your example of copying your brain to multitrack on solution to some
problem, why do you KILL the brains that fail to come up with the right
answer? That seems like, well, overkill.
> > > Q2: Do you have a right to "edit" the backup copies?
> [snip]
>
> > On the other hand, it is not only acceptable, but good to do our best
> > to act to "program" or "edit" the child's mind to make it a BETTER mind.
> > Thus, the proposed morality of mind would find that some "editing" of
> > one's own backup copies would be good, and some bad.
>
> It seems like editing (or even selecting brains), if done from the
> perspective of self-improvement would be morally acceptable if the
> information gained is more than the information lost. Destroying
> a brain that is mostly a copy that fails the "best solution" test
> probably isn't much of a loss.
Your use of the term "self-improvement" begs the question of what a "self"
is. How can you say that complete, autonomous copies of your brain are not
"selves"?
> > > Q3: If you "edit" the backup copies when they are "inactive"
> > > (so they feel no pain) and activate them are they new individuals
> > > with their own free will or are they your "property" (i.e.
slaves)?
> >
> > Without a doubt, they cannot be your slaves, whether you edit them or
not.
>
> > See the response to Q1 above.
>
> But if I went into it *knowing* that the over-riding purpose of the
> exercise was to continually and rapidly evolve a better mind, I've
> implicitly signed the waiver (with myself) that part of me is
> destined for dog food.
Again, this is the issue of moral autonomy and consent. If you consent to
such a thing - especially if it is done before and after the copying is done,
I can't question the moral right to determine the circumstances of one's own
destruction.
> > > Q4: & Greg's Extropian Exthics and the Extrosattva essay URL [snip]
> >
> > ... which is basically an extension of the Golden Rule into the arena of
> > unequal minds with the potential of augmentation, i.e. a godlike being
> > should treat lesser beings as much like itself morally as possible,
> > because those beings may one day themselves be godlike themselves.
>
> The key aspect is that you are discussing entities with the "potential
> for augmentation". There are two problems, a scale problem and an
> allowed reality problem.
>
> The scale problem has to do with the perceived degree of difficulty
> of the sub-SI becoming an SI. In theory we will soon have technologies
> that would let us provide ants (with perhaps slightly larger heads) human
> equivalent intelligence. Thus ants are "potentially augmentable". You
> don't see us treating ants with the "golden rule" generally.
I'm not so sure. What's important is not to treat all minds equally, but to
treat them as we would want to be treated if we WERE that mind. An ant isn't
capable of comprehending the notion of augmentation, but a human is. I
suppose I should reformulate what I wrote above as "unequal minds capable of
conceiving of their own augmentation".
> The allowed reality problem has to do with the fact that the SI "owns"
> the processors, presumably anything going on within them it is conscious
> of and completely controls. If the sub-SI doesn't have the interface
> hooks into the SI overlord then there is no way it can climb up the
> evolutionary ladder. Its like arguing that a neanderthal can
> build a nuclear missle or or an ant can turn itself into a human.
> These things aren't physically impossible, but the means to do
> what is required are unavailable to the subject. A sub-SI may
> be in a inescapable virtual reality in which it is a moral subject,
> but the overlord owns that virtual reality and to the SI, the sub is
> nothing but information organized to answer a question. When the
> question is answered, you are of no further use, so you get erased.
You're describing an "is"; I'm talking about an "ought". You could just as
"accurately" say that a slave in the antebellum South had no legal ability to
operate as an autonomous individual, so his owner could do whatever she
wanted regarding the quality of his life.
I say "erasing" a human-level, active consciousness is wrong because a mind
of such high order is a good in and of itself. I'm not claiming that this
leads to ethical implications that aren't difficult to work out, but it
surely leads to more acceptable results than saying a person's children are
his "property" with which he can do whatever he pleases.
> > Even without considering the potential for moral-subject-status equality,
> > though, I believe the godlike SI is not completely without constraint
> > in how it should treat such lesser beings, no more than we are in how
> > we treat lesser animals.
>
> Well, historically we set about eliminating any animals that were a threat
> and we breed animals to be relatively tolerant of our mistreatments.
> The point would be that we *breed* animals and we *grow* plants and
> to a large degree do with them what we like unless a group of us gets
> together and convinces/passes a law/forces the others that doing that
> is morally wrong and they have to stop.
Again, I think you're mixing qualitatively different kinds of analysis;
confusing description with prescription. The fact that something has been
done in the past by a lot of people doesn't make it morally right. I know
that moral nihilists claim that there can be nothing other than moral
description, that's what "right" is simply what people observe and enforce as
right. I'm doing something different here. While there may be some
fundamentally arbitrary act in choosing one or a few principles as moral
axioms, having done that, I don't accept social praxis as a determiner of
MORALITY (but it may be very important in an ethical or legal sense).
In the examples to which you point above, the subjects of selective breeding
or genetic engineering have few or none of the attributes of mind that I
attempt to raise to the level of a moral axiom. Thus they present little or
no moral challenge.
> SIs policing other SIs seems
> impossible with regard to the internal contents of what is going on
> within the SI. I can stop you from killing someone but I can't force
> you to stop thinking about killing someone.
I acknowledge that the idea I'm exploring could result in a completely new
conception of "crime". It would become fundamentally important to
distinguish between thoughts *I* am thinking and minds I am "hosting". I
don't claim to have worked out all the implications of this, but I do note
that we seem to be able to develop rules and laws for governing the
interaction of people within society, even people of vastly different
capacity and "power". We've even developed systems for governing the
interaction of systems that govern such interactions. I don's see why the
same couldn't be done with minds and minds-within-minds.
> > The morality of mind proposed here would dictate that the subprocess you
> > posit should be treated with the respect due a fairly complex mind, even
> > if
> > that complexity is far less than that of the SI.
>
> You haven't convinced me. If the scale differences are too large, then
> the sub-SI is dogmeat. Your example regarding the "easy life" for subjects
> that graudate from lab animal school works in cases where the levels are
> relatively similar (humans & chimps) but fails when the scale is larger
> (humans & nemetodes).
Quantitative scale alone isn't the only determinant: I would say that minds
"above" a certain level are deserving of being treated as moral subjects.
Finding the relevant qualitative "floor" (or more likely, "floors") is
essentially the same problem of defining mind that epistemologists and AI
researchers currently face.
> > Hal's put his finger on the fact that we're not treading entirely virgin
> > moral territory here. We already have to deal with moral questions
> > inherent
> > in interactions of unequal minds and in one person having some kind of
> > moral
> > "dominion" over another.
>
> I think the hair-splitting comes down to whether it the subject is
> externally independent (a child), or internally self-contained (an idea
> in my brain). From my frame of reference my killing my "Brhump" or
> an SI erasing a sub-SI has the same moral aspect as occurs when
> one personality of an individual with multiple personality disorder
> takes over the mind (perhaps permanently killing/erasing) one or
> more of the other personalities. I those situations, I don't think
> the psychologists get very wrapped up in the morality of killing
> off individual personalities. They judge their approach on what
> is feasible (can I integrate or must I eliminate?) and what is
> ultimately best for the survival and happiness of the "overlord".
You're right that there SEEMS to be a distinction between "child-minds" and
"minds-within-minds". I don't doubt that different ethical and moral rules
would apply, but I think one runs the risk of eroding principles of autonomy
as you move along a spectrum of "independence". I would ask how you would
articulate principles for distinguishing one from the other. I think I see a
spectrum, with a natural human child at one end, your "Brhump" in the middle,
and a completely self-contained mind-simulation at the other end of the
spectrum. But I don't think any distinctions we can make along this spectrum
solve the problem.
Let me throw you a science-fictional hypothetical. What if an SI sees your
pathetic, fragile biological nature and decides to flash-upload you while you
sleep into a section of it's own mental substrate. At first, your
environment is perfectly simulated so that you do not perceive that you've
been uploaded. Is your SI benefactor now an "overlord" as in your
multiple-personality scenario? Is it free to now manipulate your virtual
environment in any way it chooses, including to torment you? What if the
torment is for a "good cause" (whatever that might be - say, to better
understand the nature of human suffering)? What if the torment is just to
satisfy some perverse sadistic pleasure of your SI overlord? What if the
torment is just the result of random mutation of the virtual environment, and
happens because the SI neglects to keep watch over you, being preoccupied
with whatever it is that SIs care more about?
Seems like we've got exactly the situation depicted in the
Judeo-Christian-Moslem scriptures. Looks like the book of Job. It stinks.
If your answer is "That's just the way it is: He's god and you're not," then
you've basically abandoned morality entirely. I would propose to you that
that is exactly where your "scale" distinction gets you and why we better
develop an "ethics of godhood".
I've made this comment obliquely before, but will be more explicit here. One
of my intellectual heroes is Thomas Jefferson. One of the most moving
episodes in his life was his attempt to outlaw slavery in Virginia as a step
toward removing the institution from the American scene as the new republic
was being formed. When that effort failed, he wrote his famous line that "I
fear for my country when I think that god is just" - a succinct summary of
what he wrote elsewhere that his generation, by not coming to terms with the
problem of slavery, was bequeathing to a subsequent time a terrible reckoning.
I have a very similar conception of our current relationship to AI, the
potential of SIs, radical human augmentation and the development of an
"ethics of godhood". Failure to develop a rational and robust moral system
that can accommodate the real potential of transhumanism in our own time may
well condemn those of us alive 20 or 50 years hence to a tragedy not unlike
the US Civil War, but of vastly greater proportions.
> [snip re: physical realities vs. virtual realities]
>
> >
> > This is only a difficult problem if we take a simplistic view of what
> > "reality" is. "Reality" for questions of morality and ethics IS mind,
so
> > "virtual reality" is in a very real sense more "real" than the
underlying
> > physical substrate. (Phew! That was hard to say . . .)
>
> But my mind may be killing off ideas or personalities or perhaps
> imagining many horrific things and people don't generally get
> upset about it. Virtual reality is real to the people in it
> but to the people outside of it, it doesn't really exist. So SIs
> presumably don't police the thoughts of (or the virtual realities
> destroyed by) other SIs.
I don't think this distinction can hold up. It presumes a far too concrete
line between "real" reality and "virtual" reality. Consider your own present
situation. Your mind is a process "running" on the wetware of your brain,
supplied with information about the outside world through your "natural"
biological senses. What happens to your distinction between "virtual" and
"real" reality when we begin to augment those senses. The image you perceive
by way of an electron microscope is a highly "virtual" one, almost a complete
artifact of a series of "mappings". Much of your knowledge of the world
outside of the few areas you have actually visited personally comes to you
through many layers of filtration and technological intermediation.
At what point does a mind cross the line into "virtual reality". Is a "brain
in a vat", hooked up to the outside "real" world through cameras and
microphones, in "real" reality or "virtual" reality? What if the spectrum of
the camera through which it perceives the world is shifted from the "natural"
human range of visible light? What if the image presented to the brain is
augmented with textual and iconic tags conveying information about what it is
perceiving? What if some objects in the "real" world are represented to the
brain solely as icons, rather than as lines that correspond to the physical
shape of the object? What if all of the objects are? What if some objects
are represented, and others not, or their apparent size or color is modified
to convey some information? What if we then begin to replace the brain's
neurons, one at a time, with non-biological elements?
It seems that this distinction between "real" and "virtual" won't hold up. I
don't think we will cross the line in the above scenario of incremental
"virtualization" into a realm in which we can begin to treat the mind
involved as anything other than a moral subject.
> > [snip]
> > > ... (a) the relative "seniority" (?) of the actor as compared
> > > with the "person" being acted upon; or (b) the physical
> > > reality of the actions.
> >
> > I maintain that the latter factor is all but irrelevant to the moral
> > questions posed here
>
> I'm not convinced, I think the perception of the actions at the
> reality level is the critical basis for morality. Humans judge
> each others morality by what they do, not what they think. SIs
> judge other SIs by whether they destroy potential SI civilizations
> (maybe) but not whether they are erasing virtual information in
> their own memory banks.
There's no such thing as "virtual information" - bits is bits. If you've
composed a piece of music, but never written it down outside your body or
played it, is it any less a composition of music?
> > Thus, the more
> > "senior" the moral actor, the more justified we are in judging its
actions
> > (while at the same time those moral judgments become more difficult --
> > thus
> > we eventually encounter an almost christian moral inequality in which it
> > becomes nearly -- but not entirely -- impossible for "lesser", "created"
> > beings to morally judge the actions of a much "greater", "creator"
being).
> >
> I would have to agree here. If you are incapable of understanding the
> context for what is occuring in your reality, you may be incapable
> of judging the morality of *what* happens to that reality.
Again, I think you make the error of drawing a bright line where one isn't
possible. You're assuming that a subprocess of the kind we are discussing
has NO conception of the "context" of his existence. He may have an
imperfect conception of it, or one even vastly less articulated and complete
than his "host" but, above a certain (very hard to define) level, I would say
that our subject is like Job - and deserves better than Yahweh have him.
Greg Burch <GBurch1@aol.com>----<gburch@lockeliddell.com>
Attorney ::: Vice President, Extropy Institute ::: Wilderness Guide
http://users.aol.com/gburch1 -or- http://members.aol.com/gburch1
"Civilization is protest against nature;
progress requires us to take control of evolution."
Thomas Huxley
This archive was generated by hypermail 2.1.5 : Fri Nov 01 2002 - 15:05:24 MST