Re: Preventing AI Breakout [was Genetics, nannotechnology, and ,programming]

From: Eliezer S. Yudkowsky (sentience@pobox.com)
Date: Sun Oct 24 1999 - 22:09:31 MDT


Joseph Sterlynne wrote:
>
> It seems that everyone is confident that an AI of sufficient intelligence
> will be able to discover that it exists within a simulation. This is a
> little surprising. I won't say that we could absolutely guarantee its
> eternal ignorance but I don't see why an AI would not simply accept the
> laws of its environment as the laws of a nonconstructed universe.

Maybe it'd accept the laws of physics, if it wasn't smart enough to
engage in a-priori ontological reasoning. But how is the AI supposed to
accept itself? We all know that the human body and human mind are the
result of evolution, right? Conscious design would be just as obvious
to the AI.

-- 
           sentience@pobox.com          Eliezer S. Yudkowsky
        http://pobox.com/~sentience/tmol-faq/meaningoflife.html
Running on BeOS           Typing in Dvorak          Programming with Patterns
Voting for Libertarians   Heading for Singularity   There Is A Better Way


This archive was generated by hypermail 2.1.5 : Fri Nov 01 2002 - 15:05:36 MST