From: Robert J. Bradbury (bradbury@www.aeiveos.com)
Date: Sun Oct 24 1999 - 00:24:44 MDT
On Sat, 23 Oct 1999, Skye Howard wrote:
I believe that Skye may have hit upon something every
interesting that partially solves a major problem
involving self-modifying AI that has plagued the
list and worried me greatly:
> It might be interesting if you could create an
> artificial environment where you could test such
> things. For example, if you had an artificial human
> body existing on some kind of programmed level, you
> could instill these devices into it and see if all of
> the simulated functions could continue... this would
> be a ways beyond modern technology, though, because
> an artificial computer model of a genome and the full
> animal generated therefrom, not to mention more
> processing power and memory space than a medium sized
> country, might be necessary:)
The fundamental problem I fear with self-evolving AI
is the connection to the real world as we perceive it.
However, if a self-evolving AI is operating in a
simulation of the real world, then the problem becomes
much more tractable. First, the changes take place
more slowly so we have a greater ability to monitor them.
Second, if the program shows signs of modifying itself
into self-conscious sabotage of extra-environmental entities
it can be suspended/deleted.
The questions then become:
(a) Can we guarantee that the AI never discovers it is running on,
and more importantly escape from, a simulation machine?
This goes back to the entire thread of whether we can detect *we*
are running on a simulation or whether our reality is an illusion.
(b) How do we guarantee that everybody understands and
adheres to the rules that self-evolving AIs are only
allowed to exist in simulated worlds? {This is not
dis-similar from the problem of how do we guarantee
that petty dictators don't release bioweapons that
in the process of killing us, come back to "bite" them.}
I think this fundamentally comes down to a core extropian
principle involving rational thought. Rationale people
presumably seek to preserve themselves and do not undertake
operations that have a significant risk of their own death
in the process of killing others (in contrast to the
tradeoff of saving others lives at the risk of ones own life,
which may be quite rational depending on how you value the
lives involved).
This devolves into 2 basic discussions:
(a) whether an AI can discover it is running in a simulation?
(b) whether people (the irrationals) who are willing to sacrifice
themselves can/will create non-simulation environments in
which to evolve AIs.
Many thanks to Skye for providing an interesting solution to
a thorny problem and apologies to the list if this has been
hashed through before.
Robert
This archive was generated by hypermail 2.1.5 : Fri Nov 01 2002 - 15:05:36 MST