Billy Brown wrote:
>
> Brian Atkins writes:
> > I'm curious if there has been a previous discussion on this
> > list regarding the secure containment of an AI (let's say a
> > SI AI for kicks)? Many people on the list seem to be saying
> > that no matter what you do, it will manage to break out of
> > the containment. I think that seems a little far-fetched....
>
> Here's why I don't think containment is feasible for an SI:
>
> 1) No one has ever achieved it for programs written by humans. Some
> operating systems get close, but I can't think of one that has never had a
> serious security issue.
>
> 2) Supporting an SI AI would require much faster machines than what we have
> now, running much more complex programs. This makes the problem even worse.
>
> 3) An AI will be much better at programming than humans. That means that
> its efforts to get around our security will be much sneakier, and more
> complex, that those of human hackers. See #1.
>
> Even if you make a perfectly secure sandbox, we still aren't safe. Never
> underestimate the security risk posed by social engineering:
>
> 4) You have to have human/AI contact to have any idea what the AIs are like.
> This opens up lots of potential problems - the AI can talk someone into
> letting it out, bribe them to do it, 'give away' useful (or fantastically
> valuable) programs that contain seeds of itself, etc.
>
> 5) Don't forget the legal front. The AI could try to convince people that
> it is a person, and you are keeping it as a slave (not hard to do, since
> that's exactly what is happening). If it acts as its own lawyer, you're
> probably going to loose the case.
>
> 6) Do reporters ever talk to the AI? Of course they do. Think of the PR
> campaign the 'poor, helpless, exploited' AI could mount.
>
> Some of these problems are bigger than others, but that isn't the point.
> The real problem is that I thought of all these approaches in the space of
> 15 minutes, and I'm only human. What is something with an IQ of 1,000 (or
> worse, 1,000,000) going to think of?
>
> Billy Brown
> bbrown@conemsco.com
--
The future has arrived; it's just not evenly distributed.
-William Gibson
______________________________________________________________________
Visit Hypermart at http://www.hypermart.net for free business hosting!