From: Robin Hanson (hanson@econ.berkeley.edu)
Date: Mon Sep 07 1998 - 15:35:09 MDT
[In this post, I wear my participant hat. RH]
Vinge has mostly satisfied me that he does not literally mean that
post-singularity things are "unknowable," though he for some reason
still likes to use that word. It is hard to object much to his
refined descriptor, "qualitatively less understandable," and I
grant the possibility of "styles of cognition possible to higher
technologies that are radically different from ours." Vinge even
acknowledges that future self-aware modules may not be much larger
than we are.
Given this, let me focus on what I think is now the central issue:
timing.
I think that most thoughtful people will acknowledge that it is hard
to place limits on what our descendants will accomplish if substantial
numbers of them survive for another billion years, and continue to
attempt to improve their technology over this time. This implies both
that super-human artificial intelligences (AIs) may well eventually
appear, and that we now may find it very difficult to envision what
they and their world will evolve into.
In contrast, Vinge's "singularity" concept seems to me to claim much
more, so much more that it becomes a radical claim few people endorse.
It claims that these super-human AIs will appear within the next few
decades, and then within a few years transform themselves and their
world into something very difficult for us to imagine. The essential
difference between this view and the widely acceptable view I describe
above is one of timing. What most people acknowledge to be possible
over a billion years, Vinge suggests is likely to transpire within just
a few years.
Now Vinge does acknowledge the possibility of a slower transition, and
this does serve to signal that Vinge is a reasonable person. But it
does not substitute for an analysis in favor of his radical claim.
Max More and I both took issue with Vinge's timing claim, but Vinge
just refers Max More to his reply to Nick Bostrom, which is:
To me, what Nick calls "Verticality" is just a plausible side effect of
the creation of superintelligence. This is mostly by analogy with past
progress:
o Before humankind, the kingdom of life evolved by natural
selection -- a kind of "simulation" which proceeded as one with
the real world.
o Humans have the ability to run simulations internally (far more
so than other animals). Adaptation is several orders of magnitude
faster than in the previous phase.
o We humans now are developing devices which can run simulations
faster than our internal, biological "hardware" can do. I think
it's plausible that the accompanying speedup will have the
appearance of "Verticality" over the human phase.
Vinge seems to be just rephrasing this paragraph from his original paper:
... When greater-than-human
intelligence drives progress, that progress will be much more rapid.
In fact, there seems no reason why progress itself would not involve
the creation of still more intelligent entities -- on a still-shorter
time scale. The best analogy that I see is with the evolutionary past:
Animals can adapt to problems and make inventions, but often no faster
than natural selection can do its work -- the world acts as its own
simulator in the case of natural selection. We humans have the ability
to internalize the world and conduct "what if's" in our heads; we can
solve many problems thousands of times faster than natural selection.
Now, by creating the means to execute those simulations at much higher
speeds, we are entering a regime as radically different from our human
past as we humans are from the lower animals.
Before this discussion can proceed further, I think we need to get clear
on what exactly Vinge is claiming in these two passages. I'd be most
interested in how others translate these, but my reading is:
"Progress" rates increase with the speed of the processors involved.
Now it's not clear what "progress" metrics are valid here, but if
economists' usual measures are valid, such as growth rate in world
product, there are two immediate problems with this theory:
1) Progress rates increased greatly over the last hundred thousand
years until a century ago without any change in the cycle speed of
the processors involved.
2) Computer processor speeds have increased greatly over the last
century without much increase in rates of progress.
(Furthermore, even if we accept this paraphrased claim, it is not
clear that we must accept Vinge's timing claims. For this, I think
we'd need do say more about just how rapidly progress increases with
processor speed.)
Robin Hanson
hanson@econ.berkeley.edu http://hanson.berkeley.edu/
RWJF Health Policy Scholar, Sch. of Public Health 510-643-1884
140 Warren Hall, UC Berkeley, CA 94720-7360 FAX: 510-643-8614
This archive was generated by hypermail 2.1.5 : Fri Nov 01 2002 - 14:49:33 MST