From: James Rogers (jamesr@best.com)
Date: Sat Jun 02 2001 - 13:36:31 MDT
Eliezer Yudkowsky wrote:
> James Rogers wrote:
>>
>> This represents a real hard limit, for now
>> at least. Basically, you have to look at the problem as determining the
>> maximum throughput of a given algorithm (AI in this case) on a given piece
>> of hardware (A large, loosely distributed system in this case).
>
> I think that you're visualizing an AI as something like a large search
> tree, or a large neural network, or something of the sort. In traditional
> AI the AI *has* been the algorithm, but this is because traditional AI
> doesn't know how to build anything else, so they build an algorithm and
> call it AI. A real AI is a system. The system components are built from
> algorithms.
A specific organization of algorithms is itself an algorithm -- all you need
are functional criteria. The implementation of an algorithm that is simple
to describe (say, a multi-versioning transaction protocol with serializable
isolation) can be complicated to implement in practice and require many
sub-algorithms, just as an algorithm with a very complex description may be
trivial to implement in a small amount of code. The implementation of any
algorithm may have many complex parts in implementation that themselves
could be called "algorithms". An argument separating "system" and
"algorithm" in these contexts is essentially equivalent to the recent
argument separating "lookup" and "computation"; at some fundamental level
they are indistinguishable.
The one property that any AI will be required to have is a strong data
inter-dependency; I don't think this is fungible. Unfortunately, this is
the very property that marginalizes distributability. I would certainly not
say that AI is reducible to a simple search tree or even a neural network,
but a proper AI algorithm will inherently have many of the properties of
these algorithms, even if not implemented explicitly. However, I expect
that the algorithmic core of AI will be elegant and relatively small when
finally realized -- much of the componentization of AI is artificial and
unnecessary.
I am of the opinion that proper AI should be implementable with a relatively
trivial implementation. A clean implementation of an AI core should be
doable (in my experienced opinion) in <50kLoC; you can do the majority of
the checklist features, let's say Ben's list of capabilities for the sake of
argument, in a few thousand lines of C++ (realizing that the structures
created from such code are vast and extraordinarily complex).
I've actually been kind of disappointed with the means by which AI designs
have been adapted to support additional capabilities. The interminable
duct-taping of functional modules into an AI system to fix deficiencies is
pretty damn inelegant, particularly when a bunch of those functional points
could be aggregated into a single simple, but more capable, algorithm.
Duct-taping is a sign of broken theory, or at the very least broken
implementation of theory, in my opinion. Even though I am not a
particularly big fan of neural networks, there is one lesson that I've taken
from brain anatomy: Everything that needs to be done can be done by the
complex organization of simple functional units (neurons). As long as
specialization occurs within the context of organizing basic functional
units (as opposed to a network of heterogenous components), the glue logic
is built-in and maximum interaction can be realized pretty much from the
start, making the line between component and system blurry.
> If a component is built using an algorithm that doesn't scale
> well for the Internet, a seed AI can rewrite that component using a
> different algorithm that scales well for the Internet - unless, like Ben
> and I said in slightly different words, the nature of the thought itself
> is inherently such that it doesn't fit the Internet. And, as Ben and I
> continued to say in slightly different words, this just means that part of
> the AI / seed AI's psychology winds up being distributed and part of it
> does not.
This still faces the problem of computational rate limiting (Amdahl's law
basically), which will be the bottleneck whether you distribute it or not.
The only type of computation that could be distributed effectively is
processing that is almost completely out-of-band, which would necessarily
(lest you make the problem even worse) have almost no effect on the core
operation of the AI.
> The only thoughts that don't fit on the Internet are thoughts that
> inherently require low S, where S is (Anders Sandberg's?) measure of
> latency compared to clock speed. If a lot of parallel or serial
> operations are being performed on a very wide data-set, and you can't
> predict which operations will need to talk to another operation *right
> now*, then the thought has an inherent size too large to be broken up into
> packets.
You realize that you can't simply copy the data to external nodes even if
the operation could be put into small, easily distributable packets, don't
you? It is a little more complicated than this, with respect to data
integrity, even on narrow data-sets. It is doable, but you take a big hit
in the scalability department.
Cheers,
-James Rogers
jamesr@best.com
This archive was generated by hypermail 2.1.5 : Sat Nov 02 2002 - 08:07:55 MST