From: Eliezer S. Yudkowsky (sentience@pobox.com)
Date: Thu Nov 18 1999 - 16:01:10 MST
Robin Hanson wrote:
>
> Greg Burch wrote:
> >For now, let me ask you a question, Robin: Can we question that what we DO
> >want is what we SHOULD want?
>
> Sure, within a context where we care about what we want.
> That is, when it makes sense to say "I don't want to want that."
> Such things make sense when we realize that we are not atomic,
> but made of parts, spread across space and time.
Actually, you don't even need atomicity. I would divide things as follows:
1. The way things are (objective reality);
2. The way things should be (objective morality);
3. The way we currently want things to be (objective reality, part of 1);
4. The way we should want things to be (objective morality, converges
to wanting 2)
5. Our current model of the way we should want things to be (model,
converges to 4)
If 5 (your model of 4) conflicts with your model of 3, then you don't
want to want what you want.
Or in much simpler terms, and leaving out the objective morality: If
you *want* something, then you want a part of the Universe to change.
Well, what you want is also a part of the Universe, and you can want to
change what you want.
--
sentience@pobox.com Eliezer S. Yudkowsky
http://pobox.com/~sentience/tmol-faq/meaningoflife.html
Running on BeOS Typing in Dvorak Programming with Patterns
Voting for Libertarians Heading for Singularity There Is A Better Way
This archive was generated by hypermail 2.1.5 : Fri Nov 01 2002 - 15:05:48 MST