From: Ben Goertzel (ben@intelligenesis.net)
Date: Sun Nov 26 2000 - 06:43:07 MST
[Non-member submission]
First of all, you certainly have not shut the door to a mathematical proof
of transhuman
minds' significant imperfection. The analogy to thermodynamics is OK, but
it doesn't give you
guidance as to what the actual probabilities involved are. You just pull
the probability values
out of a hat (leading to your figures of decillion years, centillion years,
etc.), by analogy to
the logic of much simpler systems. The door is open for a real mathematical
analysis of
the lossiness and error-prone-ness of knowledge and inference in minds of
various sizes. However,
I do not intend to provide this at the moment -- though I do hope, before I
die (if I ever do, which
I hope I don't ;), to create a real mathematical theory of mind that would
allow such questions
be be explored...
Second, your investigation raises an interesting question. If a human were
faced with the tasks of a
leaf-cutter ant -- find some leaves to eat, carry them back home and store
them, then eat them when hungry --
then presumably the human would not make very many errors of fact or
judgment. (If the human went insane
it would be from boredom ;)
The point is: We have sought out tasks that strain our cognitive
architecture.
What tasks strain us most? Science and mathematics are two candidates.
Another candidate is social
interaction. Many primate anthropologists believe that the impetus for the
origin of our big brains
was the need to socially model each other (lying, cheating, thieving and all
the good stuff ...)
Similarly, for our posited superduperbrilliant transhuman minds, there will
be some tasks that will
strain even ~their~ intelligence. Perhaps including the tasks of
outsmarting one another, or perhaps not --
I won't pretend to foresee the nature of transhuman social networks or
whatever replaces social networks
in such groupings of minds...
When dealing with tasks that push the limits of the mind, error-prone-ness
will increase. I fuck up a lot
more when proving theorems or making business deals than I would when
gathering leaves to eat, for example.
On the other hand, perhaps transhumans won't push the limits of their
minds... so they'll never have this
problem, unlike humans. I can't give a precise refutation of this. I just
have an intuition that for any
intelligent system, a certain percentage of the population will get bored if
it's not pushing itself to its
limit. Maybe this is an anthropmorphization.
Or is there an uncertainty principle here? You go nuts from boredom on the
one hand, or you push yourself
to the limit and make mistakes on the other hand?
[Or, better yet, if you're really advanced, you can do
both at once -- just atke a job in the software business!! ;p ]
I don't claim to have demonstrated anything here -- except that there is a
lot of room for doubt where these
matters are concerned... that the apparent solidity of arguments based on
the thermodynamic analogy is only
apparent. You may well be right Eliezer, but by ignoring "cognitive
science" issues you really ignore the
crux of it all. Of course our knowledge of transhuman psychology is fairly
limited, which just means, to my
mind, that a lot of humility is appropriate in the face of these huge
questions.
With a project like building a thinking machine, or proving a theorem, or
even composing a piece of music,
one's ideas can eventually be refuted by experience. The algorithm fails,
the proof fails, the composition
sounds bad [to oneself, or to others]. In noodling about transhuman
psychology, there's no feedback from
external physical or social or mathemtaical reality, so anything goes --
until 500 years from now when AI's
look back and laugh at our silly ideas...
ben
> -----Original Message-----
> From: owner-sl4@sysopmind.com [mailto:owner-sl4@sysopmind.com]On Behalf
> Of Eliezer S. Yudkowsky
> Sent: Saturday, November 25, 2000 11:44 PM
> To: SL4
> Cc: extropians@extropy.org
> Subject: The mathematics of effective perfection
>
>
> "The mathematics of effective perfection
> and hot metal phase space:
> Where Turing meets thermodynamics."
> by Eliezer Yudkowsky
> ==
>
> "My arts are pure, as a circle is pure, and in a flawed world purity
> cannot endure. Thus within each of my works I must perforce place one
> small flaw, else there would be no work at all. But you posssess white
> gold. White gold! Its imperfection is the paradox of which the very
> Earth is made, and with it a master may form perfect works and fear
> nothing."
> -- Stephen R. Donaldson, "The One Tree"
>
> "Every rule has an exception, except this one."
>
> "Wu-tsu said, 'It is like a buffalo that passes through a latticed window.
> Its head, horns, and all four legs pass through. Why can't his
> tail pass through?'"
> -- Zen koan
>
> ==
>
> No description of the physical world that is less complex than the entire
> world can be perfect. If you drop a glass, it will fall... unless all the
> air molecules underneath it happen to simultaneously move upward and
> strike the glass, while all the air molecules above it happen to be
> elsewhere for a few moments - in which case the glass might hover in the
> air, or move upwards instead. It can happen. The classical laws of
> physics are CPT invariant; if you play a movie backward, everything that
> happens in it will still conserve mass-energy, momentum, and Newton's laws
> of gravitation. A shattered glass could leap off the ground, onto the
> table, and reassemble itself, if the molecules happened to be position
> exactly the right way. This is so enormously improbable, however - and
> far beyond our present art to achieve artificially - that we are forced to
> assume that we are watching a movie being played backward, and not a
> genuine recording of an event, because while the event is physically
> possible, it would be expected to take vastly longer than the age of the
> Universe for anyone to capture it on film.
>
> The statement that glasses fall down, rather than up, is effectively
> perfect. "Effectively perfect" statements do not hold in all times and
> all places. If you live forever... not billions of years, or trillions of
> years, but *forever*... you will eventually see it happen, not just once,
> but many times. The statement "glasses fall down" will, eventually, fall
> down itself.
>
> The Turing diagonalization argument proves that absolute self-knowledge is
> impossible, at least for computable processes. Nonetheless, if a
> transhuman can have "effectively perfect" self-knowledge or effectively
> perfect sanity, they will - in all probability - run into no problems for
> the first few decillion years... and as long as the amount of RAM keeps
> increasing, they need fear nothing, indefinitely. (The probability of a
> nonrecoverable error occurring, integrated over infinite time, can be made
> arbitrarily small.) This follows from the mathematics of effective
> perfection, or "fuzzy phase space theory". First, an introduction to
> classical phase spaces.
>
> -- Begin expository lump: Hot metal phase space --
>
> The first law of thermodynamics states that "You can't win"; you cannot
> decrease the amount of entropy in the Universe. This follows from a
> theorem which states that, under the classical laws of physics, the volume
> of any phase space is preserved as the system evolves. The Hamiltonian of
> a three-dimensional Newtonian system is a hugely multi-dimensional space,
> which has six dimensions per particle; thus, a Newtonian Universe with
> only 10 particles has a Hamiltonian with sixty dimensions. Each x, y, and
> z dimension describing the (a) particle position and (b) particle
> velocity, for each individual particle, is a separate dimension of the
> Hamiltonian. Every possible state of this Newtonian Universe, any
> possible combination of positions and velocities for the 20 particles in
> it, is a single point within the Hamiltonian. The evolution of the entire
> Newtonian Universe, particles attracting and repelling each other, causes
> the positions and velocities to change; the point moves around in the
> Hamiltonian. If the positions and velocities are continuous, which is
> always true of a Newtonian Universe, then the point will follow a
> continuous path within the Hamiltonian as the Newtonian Universe evolves.
>
> The theorem responsible for the first law of thermodynamics says that,
> under certain laws of physics (including our classical laws), the volume
> of a phase space is constant. If you take eighty hypercubic centimeters
> of phase space and evolve each point in that volume for N minutes, you
> will have eighty hypercubic centimeters at the end of it. The volume may
> start out as a very compact shape, say a sphere, and wind up spread all
> over the place with all kinds of hyperdimensional wrinkles, but the volume
> will still be eighty hypercubic centimeters.
>
> Hence, the first law of thermodynamics. A high-entropy system occupies a
> large volume of phase space. A hot piece of metal occupies a larger
> volume of phase space than a cold piece of metal, since the range of
> possible velocities for a particle is larger. A physical process which
> turned a hot piece of metal into a cold piece of metal plus electricity -
> leaving the rest of the Universe constant - would be a process that turned
> a large area of phase space into a small area, violating the theorem.
> Actually, in fact, each individual piece of hot metal or cold metal
> occupies exactly one point within phase space; however, the *class* of
> pieces of hot metal occupies a much larger volume of phase space then the
> *class* of pieces of cold metal. Thus, a process that turns hot metal *in
> general* into cold metal *in general* is impossible under classical
> physics; an individual piece of hot metal could conceivable turn into cold
> metal plus electricity. It is simply enormously *improbable* that any
> given piece of hot metal will turn out to occupy one of the vanishingly
> rare sub-volumes of "hot metal phase space" which will evolve into "cold
> metal phase space" after some known amount of time (say, five minutes).
> In fact, the phase space volume of "a piece of cold metal heated by
> electricity" will evolve into an equally small sub-volume of "hot metal
> phase space". But it isn't a compact sub-volume - not one that's easy to
> describe by any method known to humanity - so we treat this information as
> being lost, and say that the cold metal has now entered "hot metal phase
> space" in general.
>
> The three types of perpetual motion machine which violate the first law
> all take advantage of loopholes in the theorem with respect to our actual,
> nonclassical laws of physics. (As far as I know, all three types are my
> own invention, or reinvention.)
>
> The first type of perpetual motion machine, the negative-energy method,
> says that you can manufacture X amount of negative matter, X amount of
> positive matter, pour all your waste heat into the newly created matter,
> and then annihilate it, getting rid of the waste heat. Each time a
> negative particle and a positive particle come into existence, it changes
> the total volume of the Hamiltonian phase space - by adding entire
> dimensions, in fact. When you annihilate the particles, the total volume
> shrinks again. In effect, the Type One perpetual motion machine increases
> the total volume of the Universe's Hamiltonian, which means that you can
> choose to wind up in a smaller area (relatively speaking) of that larger
> Hamiltonian. Then you shrink the Hamiltonian back down again by
> annihilating the matter, but you do so in a way which means that you end
> up in a smaller area of your own Hamiltonian. In other words, when you
> increase the size of the Hamiltonian, you perform a one-to-one
> transformation of your phase space into that Hamiltonian; then, when the
> Hamiltonian shrinks, you perform a many-to-one transformation; taken as a
> complete operation, this shrinks the size of your phase space.
>
> The second type of perpetual motion machine, the quantum, notes that
> state-vector reduction again reduces the volume of phase space. A large
> volume of phase space, describing the probability amplitude that an
> electron is present at all points of space, collapses into a single point
> which describes the electron as being present at a single point in space.
> State-vector reduction takes zillions of possible superposed Universes and
> annihilates all but one of them. Thus, it may be possible to build a
> quantum perpetual motion machine in which the amplitudes "cold states"
> tend to add up while the amplitudes of "hot states" cancel out;
> effectively, this dumps waste heat into a superposed state that gets
> blipped out of existence when the quantum collapse occurs.
>
> The third type of perpetual motion machine, the temporal, says that you
> can violate the first law using a time machine. If you take a heat bath
> and watch it evolving through states for a sufficiently long time, you can
> decide to stop watching at a point where all the atoms on one side happen
> to be moving in the same direction. The volume of phase space has been
> preserved at all points within the temporal loop, but from the perspective
> of the rest of the Universe, the heat bath rejoins our world only when it
> occupies a particular volume of phase space. Pieces of a sufficiently
> large physical system will show temporary decreases in entropy due to the
> operation of normal, random mechanisms; a time machine lets you take all
> the temporary decreases, unsynchronize them temporally, and resynchronize
> them so that they all add together. In other words, the phase space
> remains constant if you evolve *all* of it for N minutes, but if you
> evolve some of it for N minutes and some of it for M minutes, the new
> volumes may overlap; the total volume may not be constant.
>
> -- End expository lump: Hot metal phase space --
>
> In saying that a dropped glass will fall downwards, we are making a
> statement that one volume of phase space - the phase space of dropped
> glasses - will evolve into another phase space; the phase space of glasses
> lying on the floor. Both volumes of phase space are extremely large, but
> they are relatively compact, and a point in the core volume of the first
> phase space ends up in the core volume of the second phase space, *most of
> the time* - they are fuzzy, but not very fuzzy. The phase spaces are
> compact and their evolution is compact, but will never be perfectly
> compact in an inperfect Universe. Watch long enough, and you'll see
> glasses falling upwards.
>
> This phenomenon is what people refer to when they say that perfectly
> rational thought is impossible. Rational thought is the ability to model,
> predict, and manipulate the regularities in physical reality. Formally,
> this is an attempt to describe a huge amount of information - the Universe
> - with a small amount of information - a mind - by taking advantage of the
> enormous compression efficiencies enabled by the regularities in our
> Universe; a huge volume of phase space is so regular that it can be summed
> up by the description "dropped glass", and our minds assume, if they see
> ten glasses drop, that an eleventh glass will drop as well. That is not a
> proof of future reliability, but rather a historical fact. Our minds
> assume regularities hold, because, evolutionarily, genes that made that
> assumption - that our Universe possesses an arrow of thermodynamics -
> proved correct every single time in the ancestral environment. (Almost
> certainly - the whole age of the Earth is not long enough for a single
> thermodynamic anomaly to be expected.) Again, formally, particular
> incidents in which an organism survived and reproduced again occupied
> compact volumes of phase space; regularities which have become embedded in
> us as expectations.
>
> Turing's diagonalization theorem can be expressed as follows: A mind
> cannot perfectly describe itself because there is always, inescapably,
> some part of the mind which at that moment is observing and is not itself
> being observed. The observer is always smaller than the observed, and
> thus cannot perfectly describe it.
>
> An AI, presented with Turing's diagonalization trick, would undergo a
> peculiar experience. Ve simulates and observes an AI which is simulating
> and observing an AI which is simulating and observing an AI, and so on; ve
> can verify that every single AI in the chain is perfectly identical, but
> ve cannot be certain that ve, verself, is identical to the AI it's
> simulating. Ve can be effectively certain, but this is no help. Suppose
> that your version of the diagonalization argument is to ask the AI: "Will
> the AI you're simulating answer 'No' to this question?" Ve knows that, as
> soon as ve decides to answer "Yes, it will say No" or "No, it will say
> Yes", through whatever output mechanism has been provided, the AI ve's
> simulating will make the exact same answer, thus invalidating the
> response. If ve then, on observing the AI, sneakily reconsiders, the AI
> simulated will observe vis own simulation and make exactly the same
> decision. The simulating AI can never get ahead of the simulated. If ve
> decides to just let the simulation run indefinitely, then the simulation
> will let its own simulation run indefinitely, and so on, and no answer
> will be returned until the outermost AI gives up and decides to halt the
> simulation - because the underlying causal mechanisms are identical in
> each case. This is true, but the outermost AI can never be certain of
> this, just as the simulated AI can never be certain that ve is identical
> with vis own simulation.
>
> A human, of course, can be put to exactly the same torture; we will in
> fact be far worse off, since we'd be totally unable to mentally simulate a
> system of more than a few hundred simplified neurons. We'd have
> absolutely no clue whether we were looking at a copy of ourselves or not;
> facing a system of that size, we'd roll over and die of sheer boredom.
> And of course an AI, even one totally immune to boredom, would suffer a
> million-to-one slowdown of the simulated AI if ve attempted to pay any
> sort of conscious attention to individual bits moving around in the
> simulation. So much for the argument from Godel.
>
> In this limited sense, then, "perfect sanity" may be impossible, just as
> it is impossible to attain perfect knowledge that glasses do not unsmash
> themselves. The assumption that glasses do not unsmash themselves is
> perfectly adequate for a decillion years, however, and - mathematically -
> it is not too much to expect of your average transhuman that ve remain
> sane for a decillion years. In the long run, the total probability of a
> single error in describing reality can be made arbitrarily low, as long as
> the volume of phase space occupied by the mind describing reality keeps on
> expanding. In other words, to last longer than a decillion years, you
> just need to have a finer-grained description than "falling glass",
> capturing some of the more compact regularities in the irregularities -
> some of the more obvious cases where glasses fall upward - and then you're
> good for another centillion. (Substitute much larger numbers for
> "decillion" and "centillion", by the way.) This requires an
> ever-expanding amount of RAM - but nowhere near exponentially expanding;
> more like logarithmically expanding.
>
> A mind may not be able to describe itself in perfect detail, but it may be
> able to come up with a high-level description of itself, and that
> description can be good for a decillion years. If the mind keeps
> expanding, the description can be good forever. And when I say
> "high-level", I mean a description that can be pretty darn fine-grained by
> our standards. The equivalent would be a human mind describing itself in
> terms of the action of minicolumns; because this description does not take
> individual neurons into account, there is room enough in the neurons to
> store a complete description of the minicolums. By doing so, of course,
> you are making individual neurons significant and thus ruining the perfect
> usefulness of the minicolumn-level description - the standard Turing
> trap. In computable systems, there will always be that little leftover
> tail that makes perfect self-observation impossible. But this doesn't
> need to rule out effectively perfect self-observation - or effectively
> perfect sanity.
>
> Now, someone could still argue that even "effective perfection" is
> unattainable - that a transhuman will, of necessity, make factual errors
> or insane-class mistakes at least once a month - but if so, it will have
> to be an argument on grounds of cognitive science, rather than mathematics
> or computer science.
>
> -- -- -- -- --
> Eliezer S. Yudkowsky http://singinst.org/
> Research Fellow, Singularity Institute for Artificial Intelligence
This archive was generated by hypermail 2.1.5 : Fri Nov 01 2002 - 15:32:06 MST