From: Jim Fehlinger (fehlinger@home.com)
Date: Thu Mar 15 2001 - 21:55:57 MST
"Eliezer S. Yudkowsky" wrote:
>
> My Standing Challenge is as follows:
>
> "Name one concrete thing that you should be able to do, but that a Sysop
> won't let you do." It can't be an intangible quality, like "having
> nanobots fully under my control" - you have to name something specific
> that you want to do with those nanobots, but which you won't be able to do
> under the Sysop Scenario.
>
> No decrement of freedom is involved.
I can think of a reason to be nervous, at least.
Presumably, the Sysop will need to be able to out-think any aggregated
subset of its clients, to make sure that no collusion or conspiracy
can subvert it. To maintain control while permitting the growth
in total computational capacity that the clients are likely to
desire (either by increasing their individual capacity or their
numbers, or both) the Sysop will not be a static entity, but will
itself be self-enhancing, for a long time (forever, maybe, if it
can find a resource like the infinitely-extensible TVC grid in
Egan's _Permutation City_).
The thing is, forever is such a long time! What if the Sysop
mutates in some unforeseen way in this abyss of time, and
requires a mid-Eternity course correction? The Sysop will always
be more rigid in one respect than the clients -- it will continue
to have to serve as their protector and mediator throughout all
time, unless of course it is programmed to cease operating after
all the clients have reached a certain level of development (or
even commit suicide, to ensure that **it** is not a threat to
the clients). This smells like the sort of thing that
eventually sent HAL round the bend. What if the Sysop starts
to get a little dotty? Egan imagined something unforeseen like
this happening in Elysium, when the Lambertians got smart
enough to TOE the Elysians out of existence.
So here are some concrete things that you might want to do that a
Sysop might or might not permit. Could you gain direct control
of your own computronium (and go hang yourself with it, if you
wanted) by forking off your own universe and leaving the Sysop
and the other clients irrevocably behind? Could you and a
friend go through the same escape hatch together? Could the clients
unanimously decide to leave together and ditch the Sysop entity?
Jim F.
This archive was generated by hypermail 2.1.5 : Sat Nov 02 2002 - 08:06:24 MST