From: Eliezer S. Yudkowsky (sentience@pobox.com)
Date: Tue Apr 02 2002 - 10:03:33 MST
ben goertzel wrote:
>
> Eli, a search for "deliberative supersystem" on Google does not turn
> anything up. So I suspect this is not a phrase you have used in your
> online writings. If you will define it for me I'm willing to discuss it
> with me.
[...]
> Would you favor us with a list of the "N Deadly Sins of AI According to
> Eliezer" ?
>
> Sins that merely cause severe illness rather than death may also be
> included.
>
> ;-)
Aaargh. Can we put this particular track of the conversation on temporary
hold until I finish proofreading the second and almost certainly final draft
of "Levels of Organization in General Intelligence", which should be Any Day
Now?
But for the record, the top three Deadly Sins are:
1) Solving problems with code;
2) Hardwiring what should be learned;
3) Physics envy ("using only one idea" comes under this heading).
Number one, in particular, may strike some people as an odd thing to say -
so a quote:
**
Programming is the art of translating a human's mental model of a
problem-solution into a computer program; that is, the art of
translating thoughts into code. Programming inherently violates the
levels of organization; it leads directly into all the pitfalls of
classical AI. The underlying low-level processes that implement
intelligence are of a fundamentally different character than
high-level intelligence itself. When we translate our thoughts
about a problem into code, we are establishing a correspondence
between code and the high-level content of our minds, not a
correspondence between code and the dynamic process of a human
mind. In ordinary programming, the task is to get a computer to
solve a specific problem; it may be an "interesting" problem, with a
very large domain, but it will still be a specific problem. The
problem is solved by taking the human thought process that would be
used to solve an instance of the problem, and translating that
thought process into code that can also solve instances of the
problem. Programmers are humans who have learned the art of
inventing thought processes, called "algorithms", that rely only on
capabilities that an ordinary computer possesses.
The reflexes learned by a good, artistic programmer represent a
fundamental danger when embarking on a general AI project.
Programmers are trained to solve problems, and trying to create
general AI means solving the programming problem of creating a mind
that can, among other things, solve problems. There is the danger
of a short-circuit, of misinterpreting the problem task as writing
code that directly solves some specific challenge posed to the mind,
instead of building a mind that can solve the challenge with
higher-level intelligence.
**
-- -- -- -- --
Eliezer S. Yudkowsky http://intelligence.org/
Research Fellow, Singularity Institute for Artificial Intelligence
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:37 MDT