From: Peter C. McCluskey (pcm@rahul.net)
Date: Tue Jun 22 2004 - 10:05:43 MDT
sentience@pobox.com (Eliezer Yudkowsky) writes:
>It is noteworthy that criticisms seem to be equally divided between people who:
>
>A) Think that humans are too awful for their volitions to be extrapolated.
>(What am I supposed to extrapolate instead?)
>
>B) Think that people's present-day decisions are just fine, and this whole
>volition-extrapolating thing is unnecessary. (Are you absolutely sure
I agree with your criticisms of these two positions, but I still have doubts.
You write as if the only differences between current humans and future versions
of those humans are due to knowledge.
But it's pretty clear to me that some differences are due to them not being
exactly the same people (i.e. identity changing over time), so that there are
real conflicts of interest between current and future instances of a given
person. The classic case studied by economists is savings rates. I wish that
the Peter McCluskey of 15 years ago had saved slightly more than he actually
did, but I can't identify any knowledge that I have today that ought to have
convinced him to do so.
It sounds like your optimization process would give more weight to the
interests of future humans than an unbiased observer would consider optimal.
This might be unimportant, but I see a small but nonzero risk that it could
do great harm to humans at one time to produce a smaller benefit to humans
further in the future.
It should in theory be possible to write into software an optimal balancing
of current and future interests that takes into account the improved
knowledge of future humans, but I don't think I could express the appropriate
goals clearly enough to write such software.
--
------------------------------------------------------------------------------
Peter McCluskey | I see no greater impediment to scientific progress
www.bayesianinvestor.com | than the prevailing practice of focusing all of
| our mathematical resources on probabilistic and
| statistical inferences while leaving causal con-
| siderations to the mercy of intuition - J. Pearl
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:47 MDT