From: Eliezer S. Yudkowsky (sentience@pobox.com)
Date: Thu Dec 03 1998 - 22:56:02 MST
We all ground in arbitrary evolutionary influences, but some systems of
thought minimize that arbitrariness and some systems maximize it.
Historically speaking, the best results, by almost any standard, derived from
reference to non-arbitrary tests.
Paul Hughes wrote:
>
> Perhaps it is arbitrary, but your supergoal produced logic is resting an equally arbitrary
> evolutionary origins. We can speculate endlessly and wildly about the differences between
> built-in human goal systems vs. supergoals produced by so-called "pure logic". I happen
> to consider myself the ultimate anthropocentric relativist. Since all we have are our
> 3-lb pieces of gray matter from which to cogitate these issues, I posit that everything we
> can contemplate is a branch of psychoneurology. Thus, lofty ideas like "pure logic" and
> the "laws of physics", are really neuro-logic and neuro-physics -which in turn are really
> neuro-neuro-logic and neuro-neuro-physics ad infinitum. Von Neuman referred to this type
> of singularity of human knowledge systems, which now bear his name. Such Von Neuman
> Singularities are essentially knowledge black holes from which our current genetic and
> neurological limitations prevent us from escaping. I see such physical/technological
> barriers to knowledge to be the greatest motivating factor for becoming a post-biological
> being.
You know, you are the first person with the courage of his conviction; to say
that morality is arbitrary, but so is reality itself. There are not degrees
of anthrocosmology (Greg Egan's name for your position); either it's all just
a dream feeding into itself, or our perceptions arise from processes as real
as the qualia themselves.
I've thought about it. Given the number of separate and distinct ways this
world could be an illusion, ranging from simulated tests of Singularitarian
honesty, to Platonic existence of consciousness, to qualia feedback loops; and
particularly given the number of times I've had to call on the Anthropic
Principle as an explanation for improbable events; it is at least 10% likely
that the entire world I know is an illusion. In which case, the illusion of
Singularity seems like the best way to bust the illusion open. Besides, it's
just a 10% probability.
Insofar as my qualia are coherent - even if coherence across time is illusory,
my perceptions of sensory qualia are NOW consistent with my qualia of memories
- I find that the least hypothesis is to assume that the causal precedents of
qualia are atoms and the observed laws of physics. Alternative hypotheses
generally do not suggest concrete courses of action, although there are
exceptions, such as the honesty-test theory.
> In the spirit of Kurt Godel, logic would seem to turn on itself, casting doubt on its
> ability to discern reality. I therefore can't help but distrust at a foundational level,
> any and all neuro^n-logical conclusions regarding any facet of Universe, including but
> not limited to the Singularity. Since in the end, my self-awareness is more certain than
> anything else, I choose to error on the side of preserving my existence rather than
> loosing it to an arguably greater intelligence. Arguably greater, in that processing
> speed and complexity will not necessarily bring about a wiser and smarter entity.
(By admitting that the Singularity has the capability to destroy your
consciousness, you are admitting that the Singularity should be treated as real.)
The problem with making recourse to infinite recursion as a source of ultimate
uncertainty is that it makes rational debate impossible - both externally and
internally. One can imagine Plato saying: "Well, that was a very impressive
demonstration that the world was composed of atoms, but insofar as we can't be
certain about anything, I choose to believe that the world is composed of four
elements, since I can observe air, water, earth, and fire more directly."
Science begins when we accept procedures for demonstrating that we are wrong;
when we admit that reality is external from us and should determine our
theories rather than vice versa. I believe in science simply because science
has resulted in technology; science displays the power to alter the world
rather than epiphenomenally explaining it, and that's what makes it relevant
to me as a means of relating choices to future qualia.
My innovation consists of applying the same admission to goals, albeit with
intelligence enhancement taking the place of technology. It's a big leap.
Nobody has ever observed a goal. But there was a time when all truth was
determined by the political clash of opinions, when postulating an external
truth was just as large a leap. And I think that intelligence enhancement
will prove as powerful as technology itself, in the end.
> Since I think we are both resting an arbitrary evolutionary influences, all I have left is
> my awareness and ability to choose. I therefore choose life over death, control and
> participation in increasing degrees of higher intelligence over deletion, self-directed
> decisions over unselfish unknowable outcomes.
One of us is right, the other is wrong, and that will determine which of these
pictures is correct, in the event of Singularity. You can choose to believe
it will happen; you cannot choose that it will happen. Unless, in the true
spirit of anthrocosmology, you believe that your expectations will determine
your perceptions.
We are both resting on arbitrary evolutionary influences, but some systems of
thought minimize that arbitrariness and some systems maximize it.
Historically speaking, the best results, by almost any standard, derived from
reference to non-arbitrary tests.
--
sentience@pobox.com Eliezer S. Yudkowsky
http://pobox.com/~sentience/AI_design.temp.html
http://pobox.com/~sentience/sing_analysis.html
Disclaimer: Unless otherwise specified, I'm not telling you
everything I think I know.
This archive was generated by hypermail 2.1.5 : Fri Nov 01 2002 - 14:49:54 MST