From: hal@finney.org
Date: Mon Oct 02 2000 - 19:45:23 MDT
Brian Atkins writes:
> Well as I said to Eugene- look around at the reality of the next 20 years
> (max). There are likely to be no Turing Police tracking down and containing
> all these AIs that all the hackers and scientists out there will dream up.
That's not clear. First, it could easily take longer than 20 years to
get superhuman AI, for several reasons:
- We may not have nanotech in 20 years
- We may hit Moore's Wall before then as computer speeds turn out to be
on an S curve just like every other technology before them
- Software may continue to improve as it has in the past (i.e. not very
fast)
- AI researchers have a track record of over-optimism
Secondly, I suspect that in this time frame we are going to see
increased awareness of the dangers of future technology, with Joy's
trumpet blast just the beginning. Joy included "robotics" in his troika
of technological terrors (I guess calling it "AI" wouldn't have let him
keep to the magic three letters). If we do see an Index of of Forbidden
Technology, it is entirely possible that AI research will be included.
Third, realistically the AI scenario will take time to unfold. As I
have argued repeatedly, self-improvement can't really take off until
we can build super-human intelligence on our own (because IQ 100 is
self-evidently not smart enough to figure out how to do AI, or else
we'd have had it years ago). So the climb to human equivalence will
continue to be slow and frustrating. Progress will be incremental,
with a gradual expansion of capability.
I see the improved AI being put to work immediately because of the many
commercial opportunities, so the public will generally be well aware of
the state of the art. The many difficult ethical and practical dilemmas
that appear when you have intelligent machines will become part of the
public dialogue long before any super-human AI could appear on the scene.
Therefore I don't think that super-intelligent AI will catch society by
surprise, but will appear in a social milieu which is well aware of the
possibility, the potential, and the peril. If society is more concerned
about the dangers than the opportunities, then we might well see Turing
Police enforcing restrictions on AI research.
Hal
This archive was generated by hypermail 2.1.5 : Fri Nov 01 2002 - 15:31:21 MST