From: Robin Hanson (hanson@econ.berkeley.edu)
Date: Wed Jul 22 1998 - 12:11:18 MDT
Damien R. Sullivan writes:
>I really have trouble believing in SI we can't understand at all.
>Things that move too fast, sure. Things that are Einstein+Feynman+Bach
>+Turing+Darwin rolled up, sure. But not the magical SI. In fact,
>magical SI through AI seems a bit incoherent; you exploit the
>Church-Turing thesis, then assume the result does something beyond
>Turing-completeness...
This whole image of a super intelligence as incomprehensible to us
as we are to ants suggests a terrible failing of our sciences of mind
(AI, cogsci, neurosci, phil of mind, etc.) Such sciences have
elaborated a great many useful concepts for categorizing and otherwise
understanding minds of humans, animals, computers, etc.
We can use these concepts to imagine a zoo of possible minds, each
with their advantages and disadvantages. With some effort, we could
envision when different types will be possible and desirable for
someone to construct, and how various selection processes might prefer
some types over others.
The incomprehensible god idea seems to me to suggest that such analysis
would be futile because it would miss the big important concepts.
While older researchers in the sciences of mind lament that each
new generation seems to reinvent the same concepts under new names,
this claim suggests that is just because the important concepts just
cannot be understood by minds like ours.
I can certainly understand the claim that it is very hard to figure
out all the details of all the possible interactions we can conceive of
between all the possible minds our current concepts can express. But
this would just suggest that, as usual, our ability to predict fades
out into a fog of possibilities. It wouldn't suggest that we not try
to shine as bright a light as we can find into that fog.
In contrast, the incomprehensible god idea suggests that before the
world gets very far filling out the zoo of possibilities we can now
imagine, it will start to create creatures we cannot now imagine, and
these creatures will be the important ones.
I have to admit I have been seduced by the apparent generality of
human language and computation abilities. We may be slow, with limited
and unreliable memory and processing. But it seems to me that given
enough of us and enough time, we can compute and understand just about
anything important. Like Damien, I have trouble believing in
"the magical SI."
Robin Hanson
hanson@econ.berkeley.edu http://hanson.berkeley.edu/
RWJF Health Policy Scholar, Sch. of Public Health 510-643-1884
140 Warren Hall, UC Berkeley, CA 94720-7360 FAX: 510-643-2627
This archive was generated by hypermail 2.1.5 : Fri Nov 01 2002 - 14:49:23 MST