From: Eliezer S. Yudkowsky (sentience@pobox.com)
Date: Mon Mar 19 2001 - 12:09:51 MST
"Robert J. Bradbury" wrote:
>
> Soooo, Samantha, Eliezer and others -- here are some questions:
Bear in mind that I am not a Sysop. Being a Friendly AI programmer
doesn't mean that you get to make the rules. These are my best guesses
about what the outcome will be under Friendliness, but things like these
are not programmer-created content.
> - "If in your godly SI or sub-SI incarnation, you choose to evolve
> forward by creating simulated copies with variations on your
> theme (presumably re-incorporating the positive results back
> into yourself) are those copies entitled to equal clock cycles?"
Okay, several oddities here:
1) "An SI can only evolve forward, or can efficiently evolve forward, by
creating entire copies of verself to test modifications." That's an
evolution-thing. It's possible to imagine results without simulating
them, and possible to simulate parts without simulating the whole. For
minds in general, forking whole copies has to be the least efficient
method of self-improvement.
2) Lee Corbin obviously has no trouble with this situation. He can fork
himself, and the new version will have full citizenship rights, but if
he's really serious in what he says, the less efficient version will
simply commit suicide and merge resources with the more efficient
version. (And not because he planned it that way or installed mental
coercions in the new copy, which might be illegal; but because he really
doesn't care.)
3) If you create a new citizen, it doesn't necessarily get equal clock
cycles. There's probably some kind of Minimum Living Space requirement -
say, enough matter that you can be a superintelligence running at full
speed and not run out of disk space until (a) the Big Crunch or (b) the
New Lands come online.
MLS might become the basic unit of currency, about equivalent to a modern
dollar. If you don't have an MLS to spare, then you can't create a new
citizen.
If computing resources are effectively infinite (or rather, indefinitely
expandable while still being finite at any given point), then scratch this
part.
> Sub-questions:
> - Are your copies granted "reproductive rights"? (Or are they sterile)?
If they're created with ten MLS instead of one MLS. Creating an entity
with the urge to reproduce using only one MLS probably comes under the
heading of child abuse; probably creating an entity with the urge to
reproduce requires a thousand MLS, or perhaps a minimum fraction (a
thousandth) of your own total resources, or something like that. (As best
I can figure it, our Solar System should contain billions if not
quadrillions of MLS for each living Terran at this time.)
> - Are your copies granted "simulation rights"? (i.e. they too can
> create evolving-forward copies that inherit the rights granted
> to their parents).
If they have an MLS to spare.
-- -- -- -- --
Eliezer S. Yudkowsky http://singinst.org/
Research Fellow, Singularity Institute for Artificial Intelligence
This archive was generated by hypermail 2.1.5 : Sat Nov 02 2002 - 08:06:31 MST