RE: When Programs Benefit

From: Hal Finney (hal@finney.org)
Date: Tue Jun 11 2002 - 20:17:58 MDT


Emlyn writes, regarding Lee's thought experiment:
> It doesn't impact on the John Clarke who pushes that button at all. The only
> consideration might be, if he believes you, that you are asking him to kill
> somebody for 10 cents.
>
> Hey, actually these are not identical runs. One of the simulations gets the
> offer to push a button. The others don't. Thus there is a relatively
> significant diversion.

No, I think the idea is that both runs are identical, up until the point
that the player pushes the button, if he does. At that point the second
run is stopped but the first run continues. If he doesn't push the
button the two runs both proceed indefinitely.

> How would the sims whose runs terminate ever experience that? It would be
> outside their experience. It's not like it'd be a personal tradgedy.
> Interesting.

Right, that's the confusing thing about this experiment. Compare it
to the simpler case where there is one run and you can push a button to
stop the run. In that situation, pushing the button would be tantamount
to instantaneous suicide. But it would still be true what you say,
that a sim whose run terminates never really experiences that fact.
The death is so instantaneous that he doesn't feel it, and he's not
around afterwards to notice that he's dead.

However I think there is still a difference between the two examples.
In the single-run case, where there is no duplication, pushing the button
will not lead to a perceptible change on the part of the sim, since it
is no longer running. Nevertheless, if we believe in the reality of
consciousness, we can say with confidence that there has been a change.
A consciousness which formerly existed has been snuffed out. Before
pushing the button, there was someone having thoughts, and afterwards,
there is not.

But in the double-run case, we can't draw that conclusion so easily, IMO.
(This is in fact precisely the issue at dispute.) Before pushing the
button, we know someone was having those simulated experiences, and after
pushing the button we know that there is still someone having experiences.
There's one copy rather than two, but there's still someone there.

It's a lot easier to distinguish between 0 and 1 than between 1 and 2 in
this situation. Between 0 and 1 is the difference between life and death.
Between 1 and 2 is the difference between life, and maybe more life.

Apparently philsopher Arnold Zuboff has written on some of these issues.
He has one of the more interesting essays in Hofstadter and Dennett's
compilation, The Mind's I. I'm going to try to track down more of
his articles.

Anyway, Zuboff uses the example of a novel. If the last copy is
destroyed, the novel is lost. But if there are two copies and one is
destroyed, the novel still exists. It's the same idea, the change from
2 to 1 is qualitatively different than the change from 1 to 0.

I'm really still on the fence on this issue. It's not clear to me that
it can ever be resolved, even in principle, even from the first person
perspective. If John pushes that button, he learns nothing about whether
anyone died as a result, and neither does anyone else. Maybe this means
that the question is meaningless. But it seems important to get it right.
As Emlyn says... "Interesting."

Hal



This archive was generated by hypermail 2.1.5 : Sat Nov 02 2002 - 09:14:44 MST