From: Lyle Burkhead (lybrhed@earthlink.net)
Date: Tue Apr 13 1999 - 18:15:26 MDT
Eliezer writes,
> Discussions of nanotechnology and AI have advanced
> far beyond the popularized simplification you call the "genie"
hypothesis;
The "genie" hypothesis is simply that AI systems with at-least-human
intelligence will work for us for free. Discussions of AI have not gone
beyond it, because it is a general idea that has nothing to do with the
details of how an AI is created.
Here is what I wrote. If you disagree with any part of this, I would like
to know what your disagreement is:
----------------------
Project: I'm going to dig a tunnel and lay a pipeline (or grow a tunnel
with a pipeline in it) from my back yard to the Saudi Arabian oil field,
and pump the oil out.
This would seem to be beyond the capability of an individual. But suppose I
have the Exxon Corporation at my command. I call the CEO into my office,
and tell him: "I am going to lay a pipeline to Saudi Arabia, as described
in this spec." At this point I give him a copy of the spec. He reads it
attentively, then looks back up at me. I continue: "Have a meeting among
your top executives this afternoon to plan the work and assign specific
tasks to each division. Apply all your resources to this task, and have the
pipeline completed in five years." He nods obediently.
The executives of Exxon do have the meeting, as instructed, then go back to
their respective divisions, and start organizing themselves for their new
task. They have to do a certain amount of retooling and retraining, since
in the past they have worked on other things besides tunnels. They also
have to hire some subcontractors. The subcontractors drop whatever else
they were doing, and work on my pipeline. All the companies throw
themselves into this task a hundred percent, and they do in fact have the
pipeline completed on schedule.
Do you see anything implausible in this scenario?
Now, suppose there are mite-sized humans (or mite-sized robots with human
intelligence). There is a one-to-one correspondence between the mite-sized
people and the employees of the corporations discussed above. They have the
same capability, on their level, that Exxon and its subcontractors have on
our level. They have equivalent tools, the same ability to acquire or
invent new tools, similar computers, the same access to outside consultants
and subcontractors, the same corporate organization, everything. They are
clones of Exxon and the subcontractors.
Let's go through the same scenario again, except this time I am not
addressing a CEO of my own size, I am addressing a mite-sized CEO. I give
him the same instructions as above, in the same peremptory tone, with the
same result.
Is scenario #2 more plausible than #1? Why? Why would a mite-sized Exxon
take orders from me, when the actual Exxon wouldn't?
If they were even smaller than mite-size -- if they were nanobots -- would
that make it any more plausible? Why would a nanoscale Exxon take orders
from me, when the macroscopic Exxon wouldn't?
Let's define a Genie as an entity with at-least-human intelligence and
sensorimotor ability, who works for free. This is distinguished from a
genie machine, which is defined as an entity that can make whatever it is
told to make, including copies of itself, with no effort on your part. The
two concepts are distinct but related -- a Genie is the AI system that
makes genie machines possible. (By the way, the word "Genie" with a capital
G does not occur in Engines of Creation -- that's just a rhetorical device
I have introduced to emphasize the absurdity of the idea of an AI system
that works for free. The expression "genie machine" does occur, however.)
Nanotechnologists assume that Genies will exist. That's what distinguishes
Drexlerian Nanotechnology from ordinary technology. The nanotechnology meme
isn't really about atoms. The key paragraphs are in the section called
Accelerating the Technology Race on page 81:
> This transformation is a dizzying prospect. Beyond it, if we survive,
> lies a world with replicating assemblers, able to make
> whatever they are told to make, without need for human labor...
>
> Eventually, some AI systems will have both great technical ability
> and the social ability needed to understand human speech and wishes.
> If given a charge of energy, materials, and assemblers, such a system
> might aptly be called a "genie machine." What you ask for, it will
produce.
Size isn't the point here. Eric Drexler's "dizzying prospect" doesn't
depend on moving atoms one by one -- that's a red herring. What makes the
prospect dizzying is the idea that entities with at-least-human
intelligence will do our bidding. The mere fact that the assemblers make
what they are told to make isn't the point either. As I pointed out in an
earlier section, we already have a system (the economy) that makes whatever
it is told to make -- there is nothing wrong with that idea in itself. The
key point here is that AI systems will design everything for us, and
therefore design will be free. (This assumption isn't stated explicitly
until page 95, but it's there all along.) The Genie saves us the trouble of
focusing our imagination and then expressing our vision in an articulate
language, including physical language, that brings about the events we
want. That's the assumption that is made throughout Engines of Creation.
That's what makes everything else possible. The implicit assumption is that
a robot could take over the roles of architect and contractor and still be
a passive machine without imagination or will -- that's what a Genie is, or
would be, if such a concept were logically coherent.
Either you have to program the robots, or you don't. If you do, then using
them to build a skyscraper will not be free of labor costs -- far from it.
On the other hand if you don't have to program them, then they have crossed
the line that separates agents from automatons. They have crossed over to
our side, and the work _they_ do amounts to the same thing as "human
labor." Either the robots are automatons, in which case you have to program
them, or they are agents, in which case you have to pay them. Skyscrapers
will never be free, because we will always have to make an effort to focus
our minds and make things happen -- or the robots will have to make the
same effort to focus _their_ minds and make things happen.
Without Genies, the ability to make things out of atoms is just an
extension of present-day technology, not dizzying at all. Molecular
manufacturing without Genies is just agribusiness.
Without Genies, there will be no sudden "assembler breakthrough." Instead
of emerging in a sudden breakthrough, nanotechnology will emerge
continuously from present-day technology, over a period of decades, step by
laborious step, each step involving an effort of concentration in a human
mind.
There are no Genies and never will be. This is a logical point, not a
technical point. It's not a question of what can or can't be done with
atoms, or what can or can't be done with computers. I'm not saying AI will
never exist. What I'm saying is that it doesn't matter -- any entity with
at-least-human intelligence (artificial or not) won't work for free. To the
extent that a robot makes independent decisions, it will have to be dealt
with as an entity that makes independent decisions. A group of robots that
could build a skyscraper by themselves would be indistinguishable from a
contractor, and would have to be dealt with as such.
I'm not saying that nanosystems will not exist, nor that they will not be
able to create large structures such as pipelines or skyscrapers. I'm
saying that a nanosystem (or any system) capable of creating a pipeline or
skyscraper will contain many human-level intelligences, and they will not
be at your command... unless you pay them, or find some other way to
motivate them.
This is an instance of the interface problem. Any agent capable of making
human decisions will have an interface comparable to the interface humans
have with each other. This is independent of the size of the agent, and
also independent of whether the agent is artificial or not.
This archive was generated by hypermail 2.1.5 : Fri Nov 01 2002 - 15:03:32 MST