From: Eliezer S. Yudkowsky (sentience@pobox.com)
Date: Mon Mar 19 2001 - 13:32:38 MST
"Robert J. Bradbury" wrote:
>
> On Mon, 19 Mar 2001, Eliezer S. Yudkowsky wrote:
>
> > The act of creation gives no moral right whatsoever to command or coerce.
> > It is simply a historical fact about the causal origins of a new
> > intelligent entity. Creators are not entirely powerless; they have some
> > control over *what* is created; but once created, the creation is a
> > citizen, and independent.
>
> Eliezer, by this statement I would conclude that you feel the
> Morovecian scenario of having the robots run simulations of
> themselves gives those robots no rights to "take back" the
> "mutations" implemented in the 'sims if the 'sims do not grant
> permission for this (assuming the robots are self-conscious).
That's quite right. If you run a simulation of yourself, it is a true
symmetrical mirror image. If you want to run a simulation and take it
back, then you had better decide on that in advance, and stick to your
decision when you become the simulation. Otherwise, obviously, you
*shouldn't* be able to do it. I don't see why you would need to. If you
want evolution, then reproduce. If you want to improve yourself, improve
yourself. Why should the twain meet, especially if it violates someone's
freedom?
Consent - volition - is everything. The Sysop Universe runs on volition.
That is the change I want to make in the nature of things; I want the
Universe to be a friendly place where volition has at least as much force
as physical law does today. I'm not quite sure what'll happen on Old
Earth after that, but it could be fairly interesting if the our massed
volitions resolve into a compromise result rather than any of the possible
extremes.
> Or does "creation" grant you complete "mind tapping" rights
> but no "directorship" rights?
>
> Another way of putting this is you develop the mutations,
> run the mutations forward to see if they are successful
> but then have to "ask permission" to access the results
> of the sim to transfer it back to the basement level.
They are not "sims". They are citizens. A citizen is not a sim unless
that is how ve chooses to define verself. You can't tap ver unless ve
wants to be tapped. Isn't that the way it should be?
> If that's the way it works, you will not find me programming
> "conscious" beings. We will all be developing entities
> that look like consciousness, walk like consciousness and
> talk like consciousness but are not consciousness under the
> legal definitions (or SysOp interpretations) of consciousness.
Again, this is exactly the way it should be. If you want to create
consciousness, then create a citizen. If you don't want to create a full
citizen with independent volition and independent rights, then you don't
get to meddle in the stuff of consciousness.
> [There are interesting parallels here with current animal rights
> movements that if carried to their logical end put a serious
> crimp in scientific exploration (or extropic creation in the
> future).]
How so? If you want to explore the configuration space of sentience, then
read CaTAI and code yourself a new citizen. If you want to run historical
simulations, use zombies. I don't see any kind of important extropic
exploration that suddenly becomes impossible. It just has to happen with
volition, in a friendly universe.
> It also suggests that from Eliezer's perspective consciousness
> is a binary state -- there are no shades of gray that have greater
> or lesser self-determination rights
You cannot create a mind so weak that you can boss it around, no. All
minds, weak or strong, are independent except as they may choose
otherwise.
-- -- -- -- --
Eliezer S. Yudkowsky http://singinst.org/
Research Fellow, Singularity Institute for Artificial Intelligence
This archive was generated by hypermail 2.1.5 : Sat Nov 02 2002 - 08:06:31 MST