Re: Uploads and betrayal

From: hal@finney.org
Date: Tue Nov 30 1999 - 17:45:40 MST


Robin Hanson, <rhanson@gmu.edu>, writes:
> I agree this is a big important change. And in an important way, this change
> makes it *easier* to envision the future. At first, we will
> replace what we currently want with what we currently want to want. And then
> we will replace that with what we then want to want, which is probably related
> to what we now want to want to want. And since we are actually pretty fuzzy
> on what we want at these abstract levels, the main result will be to create
> a lot more variety in the values that exist. Which seems harder to predict.

[To avoid typing "want to want" a great deal, I will use the abbreviation
W1 for "want", W2 for "want to want", W3 for "want to want to want", etc.]

Does it make sense to differentiate what we "want to want" (W2) from
what we "want to want to want" (W3)?

I can see differentiating simple wants (W1) from W2s. I W1 to overeat,
to lie around and read, to indulge various other bad habits. At the
same time I W2 to avoid these things and spend my time on things that
will be more rewarding in the long term. But the role of "W3" is unclear.

Earlier Robin wrote:

> Sure, within a context where we care about what we want.
> That is, when it makes sense to say "I don't want to want that."
> Such things make sense when we realize that we are not atomic,
> but made of parts, spread across space and time.
>
> So I now can want my future self to take a long term view of things,
> while my future self might rather focus on short term payoffs.
> Or my conscious mind can want my subconscious mind to not get
> so hung up on sex all the time. And so on.
>
> Saying that I "as a whole" want something is analogous to saying
> that the US "as a whole" wants something. Sometimes negotiations
> happen within the US and the US ends up taking some actions. For
> example, the US might go to war. Similarly, negotiations within
> myself can result in Robin making choices that can be interpreted
> as Robin wanting something.

If we think of ourselves as made of parts, like a Minsky Society of Mind,
then some parts W1 long-term things, and other parts W1 short-term things.
The long-term parts W1 the short-term parts not to W1 what they do,
hence they W2 long-term things as well as W1 them.

However I can't see, in this model, where W3 would come into play.
Those parts which I identify with when I say that I W2 long-term things
seemingly would be happy once this is achieved. If we got rid of (or
changed) the parts of me that W1 short-term things, I would no longer
W1 those things and hence I would W2 long-term things. That's as far
as I go. I can't see how parts of me could W3 something.

In fact I'm not sure this distinction between "W2" and "W1" really works.
Those parts of me with which I identify when I say I W2 long-term things
are the same ones which W1 those things. Saying they W2 them is just
a matter of saying that they W1 them but don't always get what they W1
because other parts of me get in the way. So really the "W2" is just
a matter of a strong "W1" which is being thwarted.

I could just as easily identify with the short-term parts of me and say
that I W2 short-term things. Sometimes people express this by saying
that they want to learn how to stop worrying and just enjoy the moment.

Ultimately choosing what to W2 is a matter of selecting specific W1s to
emphasize. The winning W1s get elevated to W2s. I can't see how to
carry this process on to produce W3s.

Hal



This archive was generated by hypermail 2.1.5 : Fri Nov 01 2002 - 15:05:54 MST