From: Eliezer S. Yudkowsky (sentience@pobox.com)
Date: Sun Jul 07 2002 - 14:02:44 MDT
Michael Wiik wrote:
> "Eliezer S. Yudkowsky" <sentience@pobox.com> wrote:
>
>> Based on my experiences in the above cases, I've created a page
>> with a suggested protocol for AI-Box experiments:
>>
>> http://sysopmind.com/essays/aibox.html
>
> I think this is ridiculous. I'm not sure what experiments per this
> protocol would prove. What if the person simulating the AI lies? The
> first lie could be agreement to the protocol.
1) The rules are set up so that the Gatekeeper has the final authority
- socially - as to what does or does not violate the protocol. I.e.,
the Gatekeeper is the one who has to send the email message saying "I
let the AI out". If this doesn't happen then under the protocol, the AI
hasn't won. If you look at the protocol, they are not rules that the AI
party could cheat on without the consent of the Gatekeeper party. If
you can get consent, that shows something in itself.
2) What does it prove? What it proves is this: Various people have
been going around saying: "Well, we'll build an AI and not let it out
until it convinces us that it's Friendly." I then assert that the AI
could hack into the surrounding humans mentally even if the AI paid
attention to any digital or physical security precautions. People then
say, "I can't imagine anything even a transhuman AI could say that could
possibly convince me if I don't want to be convinced." The point of the
AI-Box experiment is to show that they are wrong.
-- Eliezer S. Yudkowsky http://singinst.org/ Research Fellow, Singularity Institute for Artificial Intelligence
This archive was generated by hypermail 2.1.5 : Sat Nov 02 2002 - 09:15:13 MST