From: Nick Bostrom (nick@nickbostrom.com)
Date: Thu Mar 15 2001 - 15:25:52 MST
Robert Bradbury wrote:
>Reliberion's Prince of Darkness (aka Eliezer wrote):
> > (b) our civilization currently seems to be on track for either
> > extermination or Friendliness (neither future allows the amoral
> > simulation of whole civilizations),
>
>What makes you say that? We are back to the fundmental problem
>what you can do with sub-SI's. The Aristoi are a good example
>in which the elite SI's can do pretty much whatever they want
>with sub-SI's.
There are some independent reasons for thinking that we'll develop a
"singleton", i.e. a world order in which there is a single power that has
complete control and is stable. (Internally, this singleton may have many
competing interests - say, different individuals whose votes determine the
actions of the singleton.) One reason is that we might realize that a
singleton is the only way of preventing a black goo disaster. Another is
that we might understand that this degree of coordination is necessary to
avoid Robin's Burning of the Cosmic Commons scenario. A third reason is
that if there is a singularity then the transcending power might well get
enough power to become a singleton.
The problematic assumption is not that we might develop into a singleton,
but that virtually every posthuman civilization does so, and into the same
kind of singleton, i.e. one that prevents their members from running
ancestor-simulations. This requires a rather strong convergence-hypothesis
("all advanced civilization evolve in the same direction"), but it is real
possibility in my opinion, and indeed is the scenario that we should hope
is correct.
>and death, its only *virtual*. Where is the harm in that? Yes,
>you can argue that the people in the simulation don't know its
>virtual, but their opinions don't count -- they aren't real either.
Where is the harm? If you are injured, you know exactly where the harm is.
It doesn't matter whether you are living in a simulation or on the
basement-level.
It would be a fundamental mistake to think of simulations as not fully
real. Simulations are events in the physical basement universe. If the
beings in the simulations are conscious then their well-being is as
ethically important as that of those who are implemented directly in
biological brains in the basement universe.
Note that freedom of thought may not apply in unrestricted form to beings
whose powers of imagination are so advanced that when they think about
somebody that person thereby comes into existence. (If your thinking
processes involved running a detailed simulation, this could happen.) You
wouldn't be allowed to think certain thoughts for the same reason that a
human is not allowed to have a child and maltreat it. Thinking is not a
private matter when it directly affects somebody else.
Nick Bostrom
Department of Philosophy
Yale University
Homepage: http://www.nickbostrom.com
This archive was generated by hypermail 2.1.5 : Sat Nov 02 2002 - 08:06:24 MST