From: Eliezer S. Yudkowsky (sentience@pobox.com)
Date: Fri Jun 22 2001 - 20:49:57 MDT
There is no way on heaven or Earth that human flesh and synapse can build
a seed AI and never once have it interact with humans. It requires that
the seed AI be complete ab initio, that it spring into existence
full-formed. Let's call this "Minerva AI". I hereby offer an informed
opinion to the effect that building a clean-room Minerva AI is TOTALLY,
ABSOLUTELY IMPOSSIBLE pre-Singularity.
Even if you can magically build a Minerva AI, the AI can still deduce a
heck of a lot about the builders by examining its own source code. Even
if you build a seed AI, interact with it, let ver get almost up to the
equivalence threshold, then suddenly and unannouncedly freeze the
still-infrahuman AI and dump all episodic memory before locking ver in a
black box, the AI's concepts will still contain enough information to tell
ver all about humans. If you dump the concepts as well, the AI will
probably just die, assuming it hasn't already. If, somehow, the AI can
bootstrap up from from a total lack of cognitive content, and the AI has
already rewritten all of vis own source code and eliminated that source of
clues, I *still* think that the AI has enough information to take a decent
stab at guessing what the Outside is like. The source code may not have
been written by humans, but it was still written by an entity that knew
humans.
Furthermore, the higher levels of the GISAI architecture consist of
cognitive content, and the CFAI Friendship system is built on that, so
deliberately setting out to eliminate all knowledge and cognitive content
would probably leave you with an unFriendly AI even if it didn't kill the
baby.
-- -- -- -- --
Eliezer S. Yudkowsky http://intelligence.org/
Research Fellow, Singularity Institute for Artificial Intelligence
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:36 MDT