From: Daniel Radetsky (daniel@radray.us)
Date: Sat Jul 30 2005 - 15:01:53 MDT
Eliezer:
Unfortunately, I could not reply directly to your original post, as it was never
delivered to me. You might want to have that looked into. Anyhow, you wrote:
> The point of the AI-Box Experiments is that I can do 'magic' in the latter
> sense relative to some people who firmly stated that NOTHING could possibly
> persuade them to let an AI out of the box. Obviously, being human, I did
> nothing that was *strongly* magical.
But I'm not talking about the magic of sweet talking the jailer into letting
the AI out. That's another question. I was talking about the belief aired by a
number of members of this list that there is reason to believe that features of
the laws of physics might be exploited by a boxed AI. For example, Goertzel
seemed to suggest that he believed an AI might be able to use quantum
teleportation to escape the box. Perhaps I am misrepresenting him, but I think
this a fair example of the kind of belief that many people on this list hold.
> The problem of magic is the problem of a very large search space, in a case
> where we not only lack the brainpower to search each element of the space, we
> may lack the brainpower to properly delineate the search space.
Quite true.
> your inabiliity to think of a way out yourself, is only slight
> evidence that the AI will be unable to think of a way out.
I don't believe I ever claimed that that my inability to think of way out
provided such evidence. What I claimed was that there was no evidence that
there was any such reasonable way out, and hence we would be paranoid to
consider it.
> The AI-Box Experiment is a lesson in the tendency of
> sufficiently large search spaces to contain magic. That is why I refuse to
> publish transcripts. Get used to the existence of magic.
Likewise, I refuse to consider your experiment evidence until you provide your
methods. There are plenty of things you could have done which could have caused
the experiment to succeed without providing any evidence for your position. For
example, you could have paid the two guys involved to let you out. Or you could
have argued that no one would take the dangers of UFAI seriously unless they
let you out. Both scenarios count as success of the experiment, in the sense
that you were let out, but neither counts as evidence for your claim about the
existence of magic.
Of course, if you really did use such tactics to fix the experiment, you might
just publish fake transcripts. If you want to send them to me, the sooner you
send them, the more likely I'll believe them.
> The argument which Ben Goertzel cites is that, since physics has changed over
> the last few generations, we should anticipate that we have stated the search
> space incorrectly when we consider all physical means by which the AI might
> break out of the box. This does not mean that the AI *has* to go outside
> known physics to break out, because there might also be an escape route in
> known physics that you did not think of.
It is quite true that it might be the case that we have stated the search space
incorrectly, and that there might be holes in the final correct theory of
physics. It is also true that there might be holes in our existing theory that
we have not thought of. These are not arguments that there is likely to be hole
that is reasonably exploitable.
> Consider OpenBSD, the most secure OS you can obtain on an open market.
On a side note, this is a wildly controversial assertion about OSes that I
don't think that you are justified in making. You once told me in private
communication, if I recall correctly, that Exim was a Debian-based operating
system (this is false, it is an MTA). On the basis of this and other facts, I
think we ought to disregard your opinion on the matter of the security of
particular OSes.
I say this because a lot of people on this list hold your opinion in very high
esteem, and they may take your mistaken impressions as gospel and end up with
non-ideally-secure systems as a result.
> But we expect there are more bugs in OpenBSD and we expect there are more
> bugs in our model of physics...If you consider any *single* element of a large
> search space, the probability remains infinitesimal that *that single
> element* is an escape route. It is the probability of the whole search space
> that is the problem...if the AI can send arbitrarily formed bitstrings to any
> port, then the probability of a working exploit existing is high, and the
> probability of a seed AI being able to find at least one such exploit, I also
> estimate to be high...When you cite particular physical means of breaking a
> box and their apparent implausibility to you, you are simply saying that some
> particular bitstring probably does not obtain root on an OpenBSD box. What of
> it?
This seems to me to be an extremely poor analogy. When it comes to owning BSD
boxes, we have a lot of evidence that there are large families of exploits that
are both prevelant and reasonable to exploit. Buffer overflows, for example. If
there were an analogy for buffer overflows in teleporting out of physical boxes
with the physical byproducts of a processor, I might be more inclined to give
this view credence.
In the case of the AI talking its way out of the box, the position is not so
weak, as there are known prevelant and reasonable exploits in human reasoning.
> That's not how rationality works. If you don't know the answer you are not
> free to pick a particular answer and demand that someone disprove it...If
> you say that the probability of this very large search space containing no
> exploit is 'infinitesimal', you must give reason for it. If I say that the
> probability is 'certain', I must give reason for it.
Okay, mistake on my part: when I said "If I claim..." I didn't mean that
"claim" should be interpreted as having objective import. I should have said
"believe." Here's a better version: "If I believe that the probability is
infinitesimal, you have no basis to claim (i.e. with objective import) that I am
wrong." I presume that you would rewrite this "If...infinitesimal, then you may
disagree with my belief iff you have good reason to do so." I accept this. I take
it that you intend your next remarks to support the claim that you have good
reason to dispute my belief.
> To guess that physics might break down *somewhere*, or that known physics
> might contain some way to break out of the box, presumes that the blank spot
> is similar to known territory
Are you saying that in the history of physics we have often believed that
certain things were boxes and been wrong? That you could go back in time to 1650
and immediately break out of prison using only a few crude tools and your
sophisticated knowledge of modern physics?
I agree that we should accept
1. Our theory is not the final ultimate theory of everything.
2. It is possible that there exists a box-exploit in physics.
What I disagree with is that
3. It is likely that there exists a box-exploit, and furthermore a box-exploit
which is reasonable under circumstance C.
The fact that there could be a mind before which we are dog-like in our
intellectual capacity doesn't mean that (3) is true. It just means that if (3)
is true, then it's more likely that the mind will find the exploit. Also, the fact
that we have been wrong a lot in physics in the past does not support (3), only
(1) and (2).
Yours,
Daniel
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:51 MDT