From: Eliezer S. Yudkowsky (sentience@pobox.com)
Date: Mon Oct 06 1997 - 21:01:13 MDT
Eric Watt Forste wrote:
>
> Do you think that post-Singularity entities (PSEs) will not
> require scarce physical resources to carry out their projects?
> I suspect they will, and that much contemporary economic theory
> will be applicable to analyzing their behavior as a result.
I think it's more that PSEs using "contemporary" economic theory would be like
NASA using Aristotelian physics. At the very best it might be like NASA using
Newtonian physics to calculate orbits - a close approximation.
The PSEs can figure out their own damn economic theory. *I'm* working on my
program architectures.
> Eliezer's position that right and wrong objectively exist, and that
> we do not currently have adequate cognitive access to them, seems
> like a very reasonable position to me. Where he loses me is
> that part where he seems to assume that some future entity is
> more likely to have adequate cognitive access to objective
> right and wrong than we do.
Let me ask the following question: Do you think that PSEs will be able to
intuitively comprehend general relativity or quantum mechanics? Perhaps to a
human with a redesigned visual cortex, GR would be "obvious".
I am bounded on all sides by the processes I can model. Only the ability to
redesign my brain can break that kind of bound. Even Einstein could have been
said to be operating on the impetus of the Michelson-Morley experiment, giving
him the needed insight into the unintuitive operating principles of the
Universe - i.e., he "looked in the back of the book" using a physical
experiment. Maybe we could do that with respect to right and wrong - using
the objective phenomenon of the subjective experience of pleasure - but this
still strikes me as difficult.
Anyway, given that I don't have cognitive access to X, it seems reasonable to
try and extend my cognitive access before giving up. It could be that Reality
itself is closed with respect to ethics, but I have absolutely *no* way of
knowing that. I mean, my theory of ethics is based on being *completely* in
the dark. Rewiring my cognitive architecture seems like a nice place to start
- even if it doesn't work, I'll be more intelligent when it comes to figuring
out the next attempt. I can't think of any better way.
> (This is where Tipler's baroque fantasies of the Omega point lose
> me too. The Omega point looks to me like nothing more or less than
> the marriage of Heaven and Hell. Tipler goes on and on about the
> Heaven part, and never mentions the Hell.)
With all due respect to Tipler, I think the Omega Point is pure trash.
Theology, and not very good theology either. Although I liked the proof that
an infinite amount of computation can be performed using infinite time and
finite energy or infinite energy and finite time - whether the Universe is
open or closed, in other words. It might have practical applications. But
I'm not sure that's original with Tipler. And aleph-null computations doesn't
necessarily mean you know "everything knowable" - maybe it takes aleph-one.
And the phrase "everything knowable" sounds an awful lot like "the class of
all classes".
--
sentience@pobox.com Eliezer S. Yudkowsky
http://tezcat.com/~eliezer/singularity.html
http://tezcat.com/~eliezer/algernon.html
Disclaimer: Unless otherwise specified, I'm not telling you
everything I think I know.
This archive was generated by hypermail 2.1.5 : Fri Nov 01 2002 - 14:45:00 MST