From: Eliezer S. Yudkowsky (sentience@pobox.com)
Date: Wed Nov 13 2002 - 09:54:57 MST
Lee Corbin wrote:
> Here is another thought experiment that makes us less
> sure about what in general are the kinds of things
> one should do for one's own benefit.
> [snip]
> So dreadful is the punishment attending choice "10",
> however, that sooner or later you try "55". You find
> that it seems as though you escape the punishment
> despite your intellectual knowledge and despite
> the pleas and exhortations from the audience.
Maybe asking what one should do "for one's own benefit" is unnecessarily
complicating the question? I'd hit 55 with no qualms. My sanity has more
extrinsic positive value than my forgotten pain has intrinsic negative value.
On a local selfish scale I think the same would hold true. I expect to
live a long time. Again, sanity is valuable.
If I definitely knew that I were to be executed afterward and that I would
not have the opportunity to affect the Singularity in any way, one
decision would be to hit "10" until negative reinforcement destroyed that
decisionmaking process, then presumably hit 55 thereafter. Ideally one
would be able to exactly estimate how long it would take, then hit 55
until the remaining time until your execution is just long enough that you
have enough will to hit "10" on each occasion left. However, this also
begs the question; suppose that there are 110 total trials and you have
enough willpower to hit "10" ten times. I think that perhaps your will
slowly being destroyed plus 5600 seconds of pain is worse than keeping
your sanity until the end plus 6050 seconds of pain. I'd go for sanity again.
Oddly enough, the longer the trial before your execution, the more you
should choose to hit "55" throughout, since presumably your total
willpower is limited and smaller proportional differences in total pain
will tend to weigh less in your calculations.
Incidentally, I wonder how many Extropians readers are disturbed by our
having this conversation. *I'm* disturbed, but I consider it a necessary
survival skill - well worth practicing - not to flinch away from
disturbing hypothetical situations. A coherent goal system should be able
to go anywhere. Are you practicing this skill too? I admit I am
wondering why you dream up these things.
-- Eliezer S. Yudkowsky http://singinst.org/ Research Fellow, Singularity Institute for Artificial Intelligence
This archive was generated by hypermail 2.1.5 : Wed Jan 15 2003 - 17:58:05 MST