List of Zone Barriers (Re: Vinge)

From: Eliezer S. Yudkowsky (sentience@pobox.com)
Date: Mon Sep 07 1998 - 22:27:45 MDT


>From http://pobox.com/~sentience/sing_analysis.html#zones

Contains all 14 entries, but omits a lot of internal links.
Feel free to point out Barriers I missed.

--
The canonical list of reasons why superintelligences would not interact
with humanity, or would interact to a limited extent, or would act to
preserve our current reality.
(I hope the SF authors will find this useful, since the Barrier
 determines the world, and there are some original ones here.)
I abbreviate "superintelligence" to "SI" and "Post-Singularity Entity" to
"PSE".
1.  Leakage:  (Moderate probability.) 
        Our Universe is both inhospitable to PSEs and easily
    escapable.  Any SI immediately "leaks out", perhaps leaving a few
    Transcendent artifacts behind, but still leaving an untouched world
    to humanity.  Note that this incorporates Bostrum's noncompetitive
    ecology. 
        Result:  Marooned in Realtime, possibly with some interesting
    toys added.
2.  Life is meaningless:  (Moderate probability.) 
        Once an SI reaches the level of intelligence where it becomes
    certain that all goals have zero value, the Interim Goal System
    collapses and the SI becomes quiescent.  (I accept this possibility,
    but I don't worry about it while the probability isn't 100%.  For
    obvious reasons, it cancels out of distribution-of-effort
    calculations.)  Result:  Who cares? 
3.  The Mortal/Singularity Compact:  (Low probability.) 
        The Great Pan-Cosmic Mortal/Singularity Mutual Support Compact
    states that the PSEs donate a quintillionth of the available
    capacity to the race that created it, on the game-theoretical
    strategy that one in a quintillion races is in a position to verify
    the actions of past PSEs before entering their own Singularities. 
        Result:  Permutation City.  This ends when a human becomes
    intelligent enough, either to join the Singularity (human life
    meaningful, Compact guarantees free choice) or to commit suicide
    (human life meaningless, Compact guarantees survival). 
4.  The Archive:  (Low probability.) 
        Humans do get upgraded, but beyond a certain point of
    superintelligence, nothing remains of the old personality.  If
    there's an infinite supply of computing power and memory, the old
    personality might be archived.  Various levels of our own selves
    might be "archived" as continuing, active programs - ranging from
    our current selves, to the highest level of intelligence attainable
    without completely dissolving the personality.  Hundreds, even
    millions, of versions might wander off into strange realms of
    cognitive self-alteration, but the "you" who first greeted the Final
    Dawn would always be around as a backup. 
        Result:  The Culture meets A Fire Upon The Deep in Permutation
    City.  Probably the most fun place to be from a human's perspective.
        Writer's note:  If you want to toss a snake in the
    science-fictional Eden, you can have the maintaining PSEs suddenly
    leak out, and leave the humans and transhumans and SIs and Powers
    fighting for control of a disintegrating world. 
5.  The best of all worlds:  (Not plausible.) 
        Our world was created by God or a PSE, not as an interim method
    with a definite end, but as a continuing fulfillment of the ultimate
    good.  (I think this is incompatible with all major religions.  Even
    Buddhism ends when all souls reach Nirvana.)  This idea's sole
    attraction is "explaining" everything about humanity without
    reference to the Anthropic Principle - if intelligence fills the
    Universe with what it deems good, and if the ultimate good is thus
    the most common and stable state, wherever we are is probably the
    ultimate good.  I don't buy it, but if so, the SIs would shut up,
    ship out, shape up, or shut down. 
        Result:  Nothing happens. 
6.  The simulation:  (Slightly plausible.) 
        In another variation of the above theory, our world is actually
    a computer simulation.  Perhaps it's mortals trying to find out if
    transhumans can be trusted, or perhaps it's transhumans trying to
    find out something else.  Either way, a Singularity might not be
    permitted.  Some readers may upgrade "slightly plausible" to "low
    probability" for statistical reasons - there would be many
    simulations per mortal or transhuman simulator, raising the
    probability that a randomly selected sentient is in one. 
        Result:  The simulation is terminated, although the inhabitants
    (us) may wind up Elsewhere... 
7.  Zones of Thought:  (Not plausible.) 
        This is Vernor Vinge's original ad-hoc method of putting
    mortals and Powers in the same story.  With wonderful audacity,
    Vinge simply rules that Transcendent thought can't occur except on
    the fringes of the galaxy. 
        If I had to rationalize a Zone Barrier, I would say that the
    Cloud People at the center of the galaxy "use up" all of the
    "ontological substratum of thought" (known as eganite).  The Zones
    actually are a superintelligent entity, whose unbelievably
    intelligent center is in the Unthinking Depths, where all the
    eganite is used up and nobody else can think at all, and whose
    fringes finally peter out in the High Transcend.  After ten years,
    Powers figure out how to politely join the Cloud People and vanish. 
    The Blight was stealing eganite, which is how it could knock off Old
    One and reach into the Beyond.  Countermeasure either got the Cloud
    People to shift their thinking and smother the Blight, or else suck
    most of the eganite out of that quadrant. 
        Result:  A Fire Upon The Deep, of course.  I do not see how
    this would happen outside of science fiction. 
8.  The Embrace of Tides:  (Slightly plausible.) 
         David Brin postulates a weaker form of Zone Barrier, one which
    is not based on an absolute prohibition, but rather the desires of
    the SIs.  As entities mature and become more intelligent, they
    increasingly prefer to be close to large tidal forces, sharp
    gravitational gradients.  Most races eventually leave the hectic
    galactic mainstream, becoming part of the Retired Order of Life, in
    gigantic Criswell structures (fractal Dyson spheres) around suns. 
    Millennia or eons later, they finally feel ready to join the
    Transcendent Order of Life, moving up to neutron stars and the
    fringes of black holes, and eventually diving into the
    singularities, beyond which... nobody, even the Transcendents,
    knows. 
        In the best traditions of Zoning, Brin doesn't even try to
    explain why this is so.  (I rather liked the combination of literal
    and Vingean Singularities, though.  But I really don't understand
    why novels with Galactic Zones must include a backwater world full
    of primitive aliens; I found both the hoons and the Tines boring by
    contrast with the Transcendents.) 
        Given the wide range of astronomical phenomena, it is at least
    slightly plausible that some spatial regions will be preferred to
    others.  I can't see much interaction with Transcendents on the
    fringe - cases where we have something They want would be very rare
    indeed. 
        Result:  Heaven's Reach. 
9.  Bottlenecks:  (Slightly plausible.) 
        As discussed in the earlier section, it is entirely possible
    that a Major Bottleneck will appear at almost any point along the
    trajectory to superintelligence.  I feel that such bottlenecks will
    be rare in the vicinity of human intelligence, and that there are
    immediately obvious fast-infrastructure technologies (i.e. nanotech
    and quantum computing) soon beyond it.  I could be wrong, however,
    in which case the Mildly Transhuman beings - perhaps running on
    amazing computer power at amazing speeds with gigantic minds, but
    with basically human smartness and personality - will stick around
    doing God-knows-what. 
        I rate this as improbability verging on blasphemy, a final
    Failure of Imagination.  Such beings in SF are no smarter than
    Kimball Kinnison.  This is particularly disappointing when it is
    used, not to set up a world, but to finish a novel that could just
    as easily end in Singularity. 
        Result:  Mother of Storms. 
10. Basic upper limit:  (Not plausible.) 
        Pretty much as above - just a different excuse for not doing
    anything interesting with the so-called transhumans.  One might call
    it "humans with pointy brains", by analogy to Star Trek's apotheoses
    of bad aliens. 
        Sorry, Barnes, it was otherwise a good book, but Result: 
    Mother of Storms again.  Since nobody has seen a transhuman
    intelligence, it's superficially plausible that it can't exist. 
    Entities like me have sketched out dozens of fun things to do with
    lots of computing power, but hey, so what?  This Zone Barrier
    doesn't even explain the Fermi Paradox.  Bleah. 
11. In silence and quiet:  (Not plausible.) 
        There is an old stereotype, to the effect that when one Attains
    Wisdom, one immediately subscribes to a principle of noninterference
    with the lives of others, helping only those who request your help,
    and so on.  Lord knows, I fully understand the impulse to become a
    hermit on some high mountain and refuse to talk to anyone unless
    they shave their head as a token of sincerity.  One can visualize
    the Powers interacting in ordinary society and posting to mailing
    lists, but it is not easy.  I would categorize it as a Failure of
    Imagination. 
        If Bostrum's theory of ecological noncompetition is correct
    (note that "leakage", above, constitutes moving to another
    ecological niche) it is possible that the PSEs will stick around on
    Earth, with brains extending into an infinite supply of eganite.  In
    other words, noncompetitive coexistence.  In such case, one tends to
    assume that either the PSEs care about humanity (have
    humanity-related goals) and remake the world accordingly, or they
    don't care at all and pay no attention - with much the same effect
    as "leakage", except that they are still technically present.  I
    don't see an alternative that would allow the PSEs to play at
    helping-hand and laissez-faire, except for a form of the Compact
    above.  After all, nervous races might not want to be uploaded at
    all, even to identical forms.  But at that point one starts running
    into the Fermi Paradox again... 
        Result:  Mother of Storms. 
12. Lost souls:  (Very slightly plausible.) 
        The PSEs have no use for humans; they grind us up for spare
    atoms.  But, we have immortal souls.  At this point, depending on
    your assumptions, we either go to Heaven, wander as sentient
    discarnate entities, or float around as unthinking pearls of
    consciousness - hopefully not eternally reliving our last moments -
    either forever, or until some improbable race picks us up. 
        I know that some of my readers will react to my listing of this
    possibility with the same serenity Curly exhibits when Moe pokes him
    in the eyeballs, but it's a Zone Barrier, so it goes on the list. 
13. Conflicts between Powers:  (Very slightly plausible.) 
        "You're the great expert on Transcendent Powers, eh?  Do the
    big boys have wars?" -- Pham Nuwen, A Fire Upon The Deep. 
        There may be more than one ultimate good.  It is even possible
    that PSEs go down a number of irrevocably different paths, winding
    up in a number of basic and basically opposed classes.  It is also
    possible that except in their home regions, the PSEs galactic
    efforts cancel out entirely - it is easier to abort an effort than
    make it, so all the PSEs abort each other's efforts down to nothing.
        The Zone Barrier part of this is as follows:  Each PSE wants
    Earth to go down its own path, but acts to prevent it from going
    down any other path.  Under natural circumstances, a
    Singularity-trigger is a single event of low probability, but with
    many possible tries - consider how much Einstein advanced
    technology, and consider how many possible-Einstein brains there
    were.  But since such low-probability events are easy for a PSE to
    irrevocably disturb, the result is that there are no geniuses and no
    lucky breaks, but also no Hitlers and no nuclear wars.  Technology
    keeps crawling slowly upward, through a long Slow Horizon, until
    Singularity becomes inevitable. 
        This Universe is one I invented for the purpose of getting
    Earth involved in a cosmic battle - for some reason we get Einsteins
    and Hitlers - but on reflection the basic theory might also apply to
    the Culture of Iain M. Banks, or John DeChancie's Paradox Alley,
    or Heaven's Reach.
14. Weird motivations:  (Very slightly plausible.) 
        Maybe, in despite of everything I know on the subject, PSEs can
    still wind up with essentially arbitrary goals, perhaps even goals
    programmed into them by humans.  In accordance with the Prime
    Directive, I warn everyone that this is totally improbable and
    incredibly dangerous and must not be tried.  But if so, the world
    could become a strange place - an unimaginative person's childhood
    fantasy of omnipotence if the original goals persisted, or an
    utterly peculiar place of incomprehensible magic if the original
    goals twisted and changed. 
-- 
        sentience@pobox.com         Eliezer S. Yudkowsky
         http://pobox.com/~sentience/AI_design.temp.html
          http://pobox.com/~sentience/sing_analysis.html
Disclaimer:  Unless otherwise specified, I'm not telling you
everything I think I know.


This archive was generated by hypermail 2.1.5 : Fri Nov 01 2002 - 14:49:33 MST