The paradox goes like this. Omega, customarily spoken of as "a
superbeing from another planet" but here a Power or superintelligence,
arrives on Earth and performs an experiment on humans.
The experiment goes as follows. Omega puts you in a room with two
boxes, A and B. Box A, Omega says, definitely contains $1,000. Box B
may contain either $1,000,000 or nothing.
And then Omega says: "You have two choices. You can take both boxes, A
and B. But if I expected you to do this, I have left box B empty. You
get only $1,000. Or you may take only box B. If I expected you to do
this, I have put $1,000,000 in box B. You get it all.
Omega waves farewell and zooms off to his home galaxy, leaving you in a
room with two boxes.
Do you take only box B, figuring that you'll get the million dollars
that will be there?
Or do you take both boxes... on the theory that Omega has made his
prediction and left, so you can't possibly get LESS money by taking both
boxes?
(You can make Omega as accurate as you like; for example, you can assume
that he's performed this experiment on 100 people before you and he was
always right.)
Consider on one hand that fifty people took only box B and got a million
dollars, while fifty people took both boxes and got a measly thousand.
The cause and effect seem fairly straightforward.
Consider on the other hand that if you have a friend who peeks inside
the boxes, then no matter WHAT their respective contents, he will want
you to take both boxes. And since the contents of the box are
absolutely fixed while and after you make your decision - this is a
condition of the paradox - who needs a hypothetical friend to know that?
--
sentience@pobox.com Eliezer S. Yudkowsky
http://tezcat.com/~eliezer/singularity.html
http://tezcat.com/~eliezer/algernon.html
Disclaimer: Unless otherwise specified, I'm not telling you
everything I know.