From: Eliezer S. Yudkowsky (sentience@pobox.com)
Date: Tue Jan 12 1999 - 12:11:35 MST
Hal wrote:
>
> Is this notion coherent? Does it make sense to speak of a proof that
> a moral system is correct?
Does it make sense to speak of a proof that _any_ object is real?
> Earlier I proposed that a "moral system" was a method of ranking possible
> actions. Given an organism, a situation, and a set of possible actions,
> the moral system is an algorithm for ranking the actions in order (or at
> least selecting the highest-ranked action). We interpret the ranking as
> being in order of most-moral to least-moral, but mathematically it's just
> a ranking.
Let's say that a "reality system" is a method of sorting the space of
all objects into "real" and "unreal" objects. Thus the Sun would be
real, while unicorns are unreal*. How can you possibly prove something
is real without reference to a cause or perception previously known to
be real? The causal regress winds up with the Hard Problem of the First
Cause, while the perception regress winds up at the Hard Problem of
Consciousness. And yet despite this, our Universe does exist, and we do
have conscious experiences.
If you wish an example of a goal made solid, consider the qualia of
pleasure. I don't know if that's an objective morality, but it's a good
argument that the probability of objective morality is nonzero.
> Expressed in these abstract terms, there is no way to distinguish a
> "good" moral system from a "bad" one. Every ranking algorithm is a
> moral system, and they are all on equal footing. You can then
> introduce a "meta-moral system" which ranks moral systems. Given all
> possible algorithms (moral systems), it puts them into a rank order.
> Again, we would interpret this ranking as most-moral moral system to
> least-moral moral system, but mathematically it is just a ranking.
You could argue that an opinion system assigns rankings to reality
systems, and a meme system assigns rankings to opinion systems, and so
on and so on... but this is building castles on top of air when we
should be digging down. To solve the problem of consciousness, to
determine what it is about qualia that allows us to be absolutely sure
of their reality, we need to dissect cognition and the brain, not invent
observers of observers of observers. To solve the First Cause, we need
to probe the laws of physics, not invent Creators to create gods to
create worlds. To solve objective morality, we probably need to solve
the other two problems, just to grab a handhold on what morality is made of.
> I don't see how to ground this regress. It doesn't even seem to me that
> it makes sense to say that a particular ranking is objectively selected.
I don't see how to ground any of the regresses. The world doesn't make
sense to me at all. I think I'm on the wrong cognitive level to solve
these kinds of problems. You wouldn't expect a Neanderthal to solve
them; and I wouldn't expect anyone but me to solve them; and I don't
expect me to solve them.
> I'd like to see an example of an objectively-best moral system for a
> simple system.
So would I!
> Does it really seem that the problem is that we are not smart enough to
> solve this? It seems to me that the problem is simply that the question
> is meaningless.
Maybe it is meaningless. In fact, it probably is meaningless. We've
proven not only that everything is meaningless but also that we don't
exist and neither does reality. As far as we can tell the space of
choices is an indistinguishable blur, so is the space of truths, and
there's nobody to decide. So how can we choose anything, even to walk
across the room?
We're probably asking the wrong question from beginning to end. We're
lost. We don't know what the hell we're doing. We don't know what it's
all for, or why, or whether we're doing anything right, or if we're
asking the right questions, or even if we're capable of asking the right
questions. We are in a fog. We are looking through mud. We are blind.
We don't even have a visual cortex. Under the circumstances, there are
two possibilities:
1) We give up and blindly cling to our current set of priorities.
2) We start trying to clear away the fog, improve our perceptions,
write a visual cortex, raise the level of cognition.
That's the logic.
Anything more complicated is just a translation.
--
(*) = Unicorns probably are real, somewhere. Even if you specify
magical unicorns attracted to human virgins, we could probably cook some
up in a few years. The mere act of mentioning something on the
Internet's permanent record makes it more likely that some descendant
will eventually create it (even fire-breathing ducks), especially if
it's in the context of an impossibility proof.
--
sentience@pobox.com Eliezer S. Yudkowsky
http://pobox.com/~sentience/AI_design.temp.html
http://pobox.com/~sentience/sing_analysis.html
Disclaimer: Unless otherwise specified, I'm not telling you
everything I think I know.
This archive was generated by hypermail 2.1.5 : Fri Nov 01 2002 - 15:02:47 MST