From: Eliezer S. Yudkowsky (sentience@pobox.com)
Date: Sat Dec 05 1998 - 17:21:46 MST
Dan Fabulich wrote:
>
> Eliezer S. Yudkowsky wrote:
> >You have some people who say, "The choices are all the same; I'll do what
> >seems best to me." At a higher level of self-awareness, you have: "I'll
> >stick with the evolutionary system I was born in, they're all the same." At a
> >higher level of self-awareness, you say: "Which new system you choose depends
> >on how your current system evaluates that choice." Choices, systems,
> >trajectories... but I want to jump out of the system and choose the real answer.
>
> Similarly, I think there's a strong argument against being able to "jump
> out of the system" without resulting in a universe where we can't even make
> probabilistic guesses about truth and falsehood. Logic is an excellent
> example of this. We have a notion that things generally follow logically
> from one another, but this depends inherently on certain principles of
> non-contradiction, rules of inference, etc. If you jump too far out of
> such a system, you'll have no logic at all.
Well, I wrote the phrase "jump out of the system" five seconds before I had to
shut down my computer, so perhaps it deserves a little expansion. What I mean
is not necessarily that one stops having a trajectory, but that the trajectory
converges to the correct value; so that an external observer would see a
system which is "approximating the truth" rather than "moving along a
self-propelled trajectory". The system I spoke of JOOTSing (from Hofstadter:
Jumping Out Of The System) was not so much the trajectory, but a trajectory
between faceless, identical goals, without any input from the correct answer.
One can have a trajectory, and meta-trajectories, and meta-meta-trajectories,
but what I want is to approximate the truth. I want to jump from a
self-contained, self-propelled walk in a featureless space to a path
converging on the single, unique, correct point. It's the featurelessness,
the lack of external input, that requires a JOOTS.
Isn't our picture of reality only a point in opinion space? A system which
determines which values we assign to an infinite space of statements? And
don't we move in self-propelled trajectories through opinion space, determined
by our opinion of each new system for determining the truth? One can say
that, but I wouldn't. It describes the course of a Greek sophist, perhaps,
but not a modern scientist.
Because one point in that opinion space is special. One point in that opinion
space is the truth. And because we can receive, through our conscious
perceptions, messages arising from the truth; because we have learned to see
opinions as successive approximations of that truth; because we submit our
opinions to a trial by fire called "experimental verification"; the
appropriate metaphor is not a self-contained walk through a featureless
desert, but the great Quest for the Truth, the quest called Science. Opinion
space isn't flat. There are attractors in the system, spaces that repel and
attract, and one singularity called the truth.
There is no opinion space. Our task is not flitting from system to system as
the whim takes us. There is only one set of statements that matters, the true
one. Our perceptions form a gate between our minds and the truth, and our
experiments form a stronger gate.
In the space of choices, the morality Hamiltonian, we have no gate. The space
appears homogenous and flat, all probabilities and values equal to zero.
There is no way to experimentally test the value of any choice. We have no
perceptions. It is revolutionary enough to suggest that there is a single
point called the truth, that the arbitrariness is an illusion. But that first
step taken, the second task is to open up a gate, to unflatten moral space and
seek out the singularity.
I don't know how to do that. I have good reason to think it impossible to the
human cognitive architecture. So - it's time to step beyond.
--
sentience@pobox.com Eliezer S. Yudkowsky
http://pobox.com/~sentience/AI_design.temp.html
http://pobox.com/~sentience/sing_analysis.html
Disclaimer: Unless otherwise specified, I'm not telling you
everything I think I know.
This archive was generated by hypermail 2.1.5 : Fri Nov 01 2002 - 14:49:54 MST