From: Nicholas Bostrom (bostrom@mail.ndirect.co.uk)
Date: Mon Aug 25 1997 - 16:28:59 MDT
Eliezer S. Yudkowsky wrote:
> Nicholas Bostrom wrote:
> > > As an example, assume the worst scenario happens and an escaped badly
> > > programmed dishwashing nanite
> >
> > This does not seem to be the worst scenario to me. The worst
> > scenario would be something deliberately built to eliminate all
> > life. (It would be even worse if it was designed to torture it.)
>
> Very, very true. A lot of people on this list seem to lack a deep-seated
> faith in the innate perversity of the universe. I shudder to think what would
> happen if they went up against a perverse Augmented human. Field mice under a
> lawn mower.
I think I know what you mean by "the innate perversity of the
universe", but I can't think of any good way of defining or
explaining it. What would your explication be?
> > > It will spread with the speed of an bacterial
> > > infection, and be quite deadly.
> >
> > Why couldn't it spread much faster? Bacteria are limited to some
> > specifid kinds of hosts, the nanites could attack any organic
> > material and many inorganic ones too.And if they were deliberately
> > designed, they could transform themself to missiles after they had
> > eaten enough, and then swoosh accross the seven sees in a very short
> > time.
>
> I'd actually think that the infection would spread in multiple waves. The
> first wave might be small pellets travelling at hypersonic speeds, or even
> lightspeed computer viruses travelling to existing replicators. The second
> wave would be a softening-up wave that would reproduce very quickly and at
> high speed, taking small bites out of things and leaving third-wave
> replicators behind. The third wave would be immensely destructive, the actual
> gray goo. The fourth wave, if any, would assemble things out of the raw
> material thus produced.
>
> Note that these don't need to be different types of replicator. Each "wave"
> could be a different mode of action, evoked by circumstances.
Yes. It should be possible to model the first two waves
mathematically. You have a roomful of nodes, and one node is the
starting node. The starting node emits colonizers. When a colonizer
arrives at a node, that node begins to emit colonizers too, after a
certain delay time. Each colonizer can be sent to a any node. Which
colonizers do you send to which nodes in order to colonize all nodes
in the shortest possible time?
> > hope
> > that the nanites weren't deliberately designed to pile up explosives
> > on top of their bunker and blow it all away. --Yes, they *could* make
> > it, at least in a Hollywood movie...
>
> I agree, except that they'll be using nukes, not ordinary explosives. Or the
> nanites could surround the entire compound, lift it into space, and toss it
> into the Sun.
Maybe nukes, but that presupposes that they have enough intelligence
to do Uranium mining and to put together a warhead. Chemical
explosives would be easier to have them manufacture if you couldn't
give them superintelligence. --Tossing it into the sun seems a bit
farfetched and unnecessary.
> "Who will guard the guardians?" - maybe nanotechnology would give us a perfect
> lie detector. Nanotechnology in everyone's hands would be just like giving
> every single human a complete set of "launch" buttons for the world's nuclear
> weapons. Like it or not, nanotechnology cannot be widely and freely
> distributed or it will end in holocaust.
Yes, yes. At least in the absense of a working immune system, but we
have doubt's that such is possible. Do you have any concrete reason
why it could not work, though?
>Nanotechnology will be controlled by
> a single entity or a small group... just as nuclear weapons are today.
Right. Though see my Safe Libertarian Future scenario in my reply to
Anders. Would you agree that it would be a stable state? (I don't
claim that it is likely to happen.)
> If that entity is benevolent and Libertarian, utility nanites would be
> released as they were programmed - to eliminate hunger, starvation, old age,
> death, etc. The world would remain much the same, except most forms of
> physically based pain and coercion would be eliminated. Other utilities might
> be more flexible. No utility will give access to the forbidden molecular
> level, but many might give access to higher levels. People might be able to
> edit their synapses or their tissue-level body structure. (The former
> scenario might result in Singularity in fairly short order.)
Yes. Never forget to mention the psychoactive drugs that will be
possible though!
> If that entity is malevolent, immediate and indiscriminate use of nuclear
> weapons would be free humanity's only hope of survival. Humanity can survive
> nuclear war and fallout. It cannot survive molecular warfare.
That's unfortunately the way it looks.
------------------------------------------------
Nicholas Bostrom
bostrom@ndirect.co.uk
*Visit my transhumanist web site at*
http://www.hedweb.com/nickb
This archive was generated by hypermail 2.1.5 : Fri Nov 01 2002 - 14:44:46 MST