From: Doug Bailey (Doug.Bailey@ey.com)
Date: Thu Sep 17 1998 - 08:25:19 MDT
The central idea behind my post regarding the ultimate potential
of SIs and HIs is to analyze the "unknowability" and
"incomprehensibility" traits used by some to describe a post-
Singularity environment. Using the "weak SI" and "strong SI"
distinctions Nick mentioned, if the post-Singularity universe is
populated by weak SIs then it will be comprehensible to us. If it
is populated by strong SIs then it will be incomprehensible.
The question of whether SIs will be "strong" or "weak" does not
appear to be a question of SI capabilities but instead should be
an inquiry into the nature of knowledge. Is there knowledge to be
had that is forever beyond the grasp of human intelligence (even
amplified HI)? Its a epistemological question which I doubt can
be resolved easily. Such knowledge would be beyond our abilities
of recognition, inference, deduction and modelling. We can not
speculate on the feasibility of the existence of such knowledge
since, by definition, we can not appreciate the validity of the
knowledge. Why should we assume such knowledge exists except out
of some ill-advised attempt at anthropic humility?
If a SI is a strong/qualitative SI then it will be able to access
these areas of knowledge that HIs can not. This type of situation
parallels the turbo dog brain example Nick described. HI is
qualitatively superior to >DI because of HI abilities of cognition,
deduction, modelling, inference, etc. However, if we modified the
>DI's software then it might achieve HI levels. If a SI is only
a weak/quantitative SI then it might think faster and more
efficiently, but it would not be able to access those areas of
knowledge unknowable to HIs. The weak/quantitatively-superior SI
would reach milestones faster but it would not reach more milestones
overall.
The Knowledge Question raises an interesting issue. Even if
Strong SIs existed, how would we properly recognize them as Strong
SIs? Since the knowledge they accessed that heralded their
ascension to Strong SI status is beyond our ability to appreciate,
how could we distinguish between a Strong SI and a Weak SI that
has lost its marbles? When faced with such a choice it appears
that our only guidance becomes the faith we have in the SI. Either
we say, "Gee, what the SI is doing now is completely beyond me.
Since its a SI, it probably knows what its doing. Its a Strong SI"
or we say, "This thing has gone nuts." Another problem is
differentiating between knowledge that is forever beyond us and
knowledge that is beyond us for now but will be understandable
after another 200 years of scientific work.
Doug Bailey
doug.bailey@ey.com
This archive was generated by hypermail 2.1.5 : Fri Nov 01 2002 - 14:49:35 MST