From: Ben Houston (ben@exocortex.org)
Date: Mon Jun 11 2001 - 20:18:15 MDT
Hi Durant,
The Law of Requisite Variety
As a possibly useful side-note I would point you to "the law of
requisite variety" of the cybernetician Ross Ashby. This law concerns
the relationship between to interacting systems. It states that for
system A to regulate (i.e. control) another system B, system A must have
at least the variety of system B. In other words the more adaptive
controls the less adaptive...
References:
http://pespmc1.vub.ac.be/REQVAR.html
http://pespmc1.vub.ac.be/ASC/LAW_VARIE.html
Cheers,
-ben houston
http://www.exocortex.org/~ben
-----Original Message-----
From: owner-sl4@sysopmind.com [mailto:owner-sl4@sysopmind.com] On Behalf
Of Durant Schoon
Sent: Monday, June 11, 2001 3:15 PM
To: sl4@sysopmind.com
Subject: Coercive Transhuman Memes and Exponential Cults
In March, Eliezer mention the notion of "coercive transhuman memes" when
discussing another issue:
> Date: Tue, 20 Mar 2001 05:02:56 -0500
> From: "Eliezer S. Yudkowsky" <sentience@pobox.com>
> To: sl4@sysopmind.com
> Subject: Re: How To Live In A Simulation
[snip]
> What do I mean by a "commonsense" solution? I mean one in which
> second-order moral imperatives are applied with room for slack, rather
> than being absolute. By "second-order", I refer not to basic
> Friendliness, but to consequences of Friendliness - for example, the
idea
> that Old Earth needs to be kept free of coercive transhuman memes, and
> thus the idea that transhumans and humans shouldn't interact.
[snip]
My question is: What boundaries can we expect to be formed when
intelligence
goes through the roof? I'm sure this is a question to be answered by an
SI
or Sysop, but maybe someone would venture some interesting speculation
in the
mean time.
Imagine transhuman Bill becomes vastly more intelligent than person
Alan. So
much more that Bill can easily influence Alan to do things Alan might
not want
to do on his own.
Fortunately, the Sysop is there to protect Alan by advising against
actions
which might have deleterious outcomes for him.
Now, imagine Charlie. Transhuman Charlie is sufficiently more
intelligent than
transhuman Bill. Again our favorite Sysop can warn Bill off from doing
any
thing he might not want to do, if only he knew all the consequences. But
there
is a new twist with this case. Transhuman Charlie might convince
transhuman Bill
to modify vis own volition so that accepting new memes from Charlie is
highly
desired. Transhuman Charlie might advertise his suggested memes cleverly
so that
transhuman Bill bill chooses them with no violation of vis volition.
The result? Exponential cults win (in a weird way).
Possible?
Cults do not violate volition, do they?
Will there be protections instead? Classifications of Transhuman
Intelligence?
Level N may not try to influence anyone below Level N....
These seem to me to be the interesting kinds of problems that arise once
you've
chosen volition as the "be all, end all" metric of the universe (and
yes, I
currently subscribe to this view).
-- Durant Schoon
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:36 MDT