From: Eliezer S. Yudkowsky (sentience@pobox.com)
Date: Tue Aug 26 1997 - 15:25:10 MDT
Anders Sandberg wrote:
>
> "Eliezer S. Yudkowsky" <sentience@pobox.com> writes:
>
> > Cycles:
> > 1A. Programmer writes a piece of code to do something.
> > 2A. Code evolves into something small, dense and tight.
> > 1B. Code leaps into new evolutionary space and discovers new algorithm.
> > 2B. Programmer expands algorithm and generalizes it.
>
> Sorry, step 2A doesn't occur that easily. GAs tend to produce
> bloated, redundant and weird code; their strength is that they are so
> good at finding flexible solutions.
WHAT? What about all those famous stories where a GA does in 28 steps what a
human could only do in 31? GAs produce code that's tight where ours is
bloated, and bloated where ours is tight. Hence the double cycle.
> The basic idea above is sound, Thomas Ray has suggested something like this
> to get completely unexpected solutions to problems nobody has yet posed.
>
> > How much time would it take for a nanomachine to construct a nuclear weapon?
> > I think we can assume that nano is at least as destructive as nuke.
>
> Numbers, please. It is easy to claim something like this, but is it
> really true?
I don't know. But you do:
> Building a nuke: you need around 10 kilograms of uranium 235 (or whatever
> isotope it was). There is around 2 grams uranium / tonne in the crust of the
> earth, of which 0.72% is U235, so to get 10 kg you need to process around
> 7000 tonnes of crust. I'm not sure how much energy is required to reduce
> the UO2 to pure U, but it is a noticeable amount (are there a chemist
> in the house?). Assuming the nanites cover a large patch with solar
> collectors, they can get around 500 W/m^2, which has to cover their
> replication, search through the crust, reduction, isotope separation and return
> to the "base"; how much energy this is is a bit hard to tell right now
> (it is 1.30 in the morning here :-), but it looks like it will take
> a while for the bomb-mold to blow up. A wild guess would be around a
> week.
Well, there's a couple of questions I'd pose about nuclear nanotech:
1) Can mini-assemblers build mini-thermonuclear devices? If you have 10
grams instead of kilograms, but you still put the deuterium at the center and
jacket with U-238, could you get the same effect? What about custom-assembled
crystals that will collapse easily and tightly... maybe laced with water to
slow neutrons down?
2) What about direct fusion devices? Inertial confinement would be my guess.
Take the deuterium and imprison it inside a fullerene-bracketed block of
diamond, then apply a large energy pulse, perhaps from an enormously efficient
quantum-well laser.
3) Given either of the two above, could you build a REALLY HUGE thermonuclear
bomb, perhaps by surrounding the item with a huge U-238 crystal? Direct the
blast in a specific direction using the properties of crystals, maybe with a
bit of quantum mechanics mixed in? Use the blast to pump a graser?
In other words, nanotechnology operates as close to the level of nuclear
reactions as we operate to the level of chemical explosives. I can't help but
think that there will be similar improvements in technology. Shaped charges.
Fuel-air-explosives. You get the idea. Only some tiny fraction of atoms fizz
or fuse. Maybe with a properly shaped crystal, perhaps utilizing a few
quantum properties to guide neutrons or arrange collisions, you could improve
that by an order of magnitude.
I speculate that nanotechnology will result in an improvement in nuclear
weapons comparable with the superiority of TNT over gunpowder, for similar
reasons. And while some of the quantum calculations above may seem rather
numerically intense, remember that the calculations will probably be done on
quantum computers.
> Doing the same work as a nuke with nanites (i.e. disassembling everything
> within a few hundred meters and blasting everything within a few kilometers)
> is rather tricky, since it is extremely energy intensive. You need plenty
> of energy to do the disassembly (essentially you have to break most molecular
> bonds), and nanites are bad at making blast waves.
I don't see why. First of all, they'll be able to construct explosives
completely to order. But let's be conservative and figure that they use TNT.
I'm far from an expert on explosives... but couldn't you get a bit of an
improvement by detonating the ENTIRE bomb with electronic synchronization, so
that the entire blast arrives time-on-target? A shock wave is just that, a
wave. So modern TNT is a huge splash made up a lot of little uncoordinated
splashes, while nanotech TNT is the precisely synchronized sum of all those
splashes. Almost exactly like the distinction between a light bulb and a laser.
> I would like to do some more careful calculations on this (or rather, that
> somebody with the right knowledge did it instead of a bumbling amateur like
> me), but it is IMHO clear that we should not be overly worried about
> nano-built nuclear weapons but rather nano-ebola. Everything in the world
> has limits set by the laws of nature.
Like, say, lightspeed? Or thermodynamics? My "workarounds" page for things
like that is still up. Again, I'm trying to point out that a bit of
imagination is necessary to see the enormous destructive potentials here. I
think we can safely extrapolate a *little* bit beyond current capabilities.
Most of what you're saying has nanotech being less dangerous than modern
technology. I can't help but compare this to the old saw about "Atomic
technology may someday equal the capabilities of present bombs, but it is
unlikely to produce anything more dangerous." [Paraphrase.]
> > Remember that the Bad Guys aren't operating under the same restrictions as the
> > Good Guys. The wannabe dictators will use directed evolution on a scale no
> > Good Guy would dare to contemplate. And evolving mutually competing predators
> > will go a lot faster than mutually supporting immune systems.
>
> Ever thought about the fact that the immune system isn't mutually supporting?
> It is rather a balance of predators which do not predate on each other since
> that destroys their fitness.
Interrogative? Expand, please.
> Good guys will be motivated to contemplate directed evolution since the bad guys
> do it; and they tend to have more brainpower and money on their side. I
> think the weapon of openness is important here.
It could go either way. Maybe the bad guys kidnap all the scientists and then
use pain-center stimulation to achieve faster results. Idealism may win
battles and break ties, but it is not a defense to be trusted.
> > Our immune systems are unimaginably more sophisticated than a virus or a
> > bacterium, using controlled evolution to combat natural evolution. And yet we
> > still suffer from colds and diseases. The only reason that the viruses
> > haven't killed us outright is that it isn't good strategy.
>
> Exactly. So the major question is: is it possible to create a nanite
> infection that is deadly (or subtle) enough to wipe out all competition?
> Don't reflexively answer 'yes' to it, try to give a considered answer
> of why it is likely (or why not). Note that it would have to be able
> to spread across the world fast enough to subvert everyone, or circumvent
> all attempts at detection. That is assuming a lot.
Alternatively, it might only have to take over a single power base. Your
question assumes that the goo, once detected, is doomed. My version assumes
that the goo, once established, is undefeatable. This is pretty much what
we're arguing.
> > I want to repeat this, because it's important. Our immune systems are the
> > closest analogue to proposed nano-immunities. The mismatch in available power
> > and sophistication is enormous. Our immune systems learn from experience, use
> > controlled and directed evolution, have memories... everything but the ability
> > to consciously design things. And yet the viruses waltz casually through our
> > bodies, because it's so much easier to destroy than create.
>
> Have you studied medicine? Then you know viruses certainly do not waltz causally
> through your body (of which around *one kilogram* apparently is immune cells, if
> you are a normally built male).
>
> > It is easier to destroy than create!
>
> You repeat this as a mantra. And of course you have the second law of
> thermodynamics on your side. The problem is that you do not attempt to
> make quantitative comparisions between the strengths of different systems,
> and instead rely on plausible-sounding arguments. That is definitely
> *not* a good strategy if you are trying to discuss something important
> where we do need a well planned policy.
How can I make quantitative comparisions between systems that don't exist?
I'm just trying to point out the qualitative causal forces at work. When
we're finished cataloguing, we could try to set up a quantitative comparision,
bearing in mind that it would be utterly false-to-fact and useful only for
speculative purposes.
> > If any human is even capable of designing an immune system,
> > then the average educated person will be capable of breaking it, given time
> > and effort. Any twisted genius will go through it like tissue paper.
>
> Sigh, here we go again. What is your evidence for this? A few months ago
> somebody on sci.nanotech posted a challenge for people to come up with
> nanotech immune systems and he would find flaws in them. I don't think
> there was any follow-up, which is sad, but it might be worth taking up
> here on the list. Unfortunately I do lack the time to deal with this
> as carefully I would like, since I consider this very important, but
> let's try a simple sketch to see how easy it is to vanquish a designed
> immune system:
Yes, let's!
> The body is surrounded by an inert skin (say diamond);
Isn't diamond something that can be set on fire at sufficient temperatures?
Isn't the crystal fragile when properly angled forces are directed against it?
How much of an insulator is it for temperature? What about gamma rays?
> attempts to
> physically breach it can be detected from the inside and the surrounding
> region sealed.
Sealed by what? If it's better than diamond, why not surround the whole body
with it? If the sealant is worse than diamond, or is diamond, why wouldn't
anything capable of penetrating the external wall go right through the sealant?
What detector mechanism are you using? Can I falsely trigger it over the
entire body, thus making the "detector" useless and redundant? If sealed-off
areas are easier to attack, wouldn't the false alarm let me peel off an entire
layer? And why couldn't I just repeat the action until you were the size of
an apple core?
> The rest of the organism (could be a transhuman, factory
> or a city) is compartmentalized with similar walls; infected sections
> can be isolated.
Okay, so you've made the problem of immunity recursive on immunity for a
single section. This may actually be an excellent plan of defense, allowing
for partial breaches and reserves. But a design for an immune system it's not.
> Immune devices move around, interrogating "cells"
> (subsystems) by comparing their surface markings with allowed types
> (this list can be kept secret from someone who disassembles a device
> by using a trapdoor function), and occasionally disassembling the
> cell to check its innards.
Okay, I see a trend developing in your plan. Detection is the important
thing; it's assumed that pockets of infection, once detected, can be
obliterated. Very much like the human immune system. But if viruses could
outfight white blood cells, we'd be dead. I'll conceed that fooling detection
might be made difficult, almost impossible, through the use of digital
signatures. But again - how do you keep a single cell from being penetrated?
Once a cell is penetrated, you destroy it - but does the destruction keep the
infection from spreading? A single spore might be defeated by this strategy.
A sea of gray goo won't be.
> Other immune devices check the general
> activity, looking for deviations from the normal state (there is
> a paper on the net somewhere about using GAs to evolve anti-hacking
> agents, this is something similar).
No comment. Part of it isn't very complicated, given digital signatures.
Nor, as far as I know, can a nanotech penetrate a wall invisibly. I'll
concede you perfect detection. For this discussion, it's the fight I care about.
> If a compartment is found to be
> infected it is terminated with extreme prejudice (the human body
> uses the granulocytes and their nasty cocktail of free radicals,
> chlorine acids and proteolytic enzymes, this would use something
> worse or simply throw out the infected compartment from the
> organism).
Why shouldn't the infected compartment terminate a nearby one with extreme
prejudice, and do it first? As far as I can tell, your strategy simply lets
me destroy your entire complex cell by cell. Penetrate a cell, watch it go up
in flame... repeat.
> Note that production of new immune devices can be done
> by assemblers in well defended deep regions, and that the
> devices themselves can be fairly simple since they do not need
> to replicate, just find threats.
And the gray goo can do likewise... but it has more resources, since it
surrounds the city.
> So, what are the obvious holes?
The real obvious one is this. Your immune system - any immune system
conceived along these lines - requires that two nanosystems fight each other
to a standstill. If two nanosystems fight and destroy each other, then any
city in an island of gray goo is doomed. The goo simply makes repeated
attacks, and each time, the city shrinks a little. We won't even speak of
such horrors as cutting off that city's solar power.
In short, the immune system requires that *defense* have an advantage over
*offense*. I can conceive of any number of ways that this could happen. I
shall now proceed to list them.
The most obvious way is to have a symmetrical battle. Two systems battle.
They destroy each other. A small no-mans-zone is created. But then, both
city and sea expand into the zone at the same rate, gobbling atoms as they go.
If more nano is required to destroy than to defend, even by a small amount,
then the city actually builds up more raw material as it goes along. Maybe
ditto for energy reserves. Depends on the details of the molecular landscape.
Then there's the barrier whose surface is stronger than the amount of energy
that could be concentrated on any one point. There are two opposing forces
here. First of all, the barrier is dispersed and the energy is concentrated.
But the barrier is deep, while the energy might be a single pulse. Depends on
the details of the molecular battleground. A miniature swords-vs.-armor thing.
Moravec has a great discussion of exotic forms of matter(1). "Higgsium" is
1e18 times as dense as water; "Monopolium" is 1e25 times as dense. My
impression was that Higgsium might be hard to penetrate, even using Higgsium
projectiles; only Monopolium would suffice. With Higgsium and without
Monopolium, then, any area taken might be easy to hold onto. With Monopolium,
the reverse is true.
Similarly - and here I speculate without knowledge - there might be a way to
give a material an unbreakable "quantum surface tension". Or subbaryonic
picotechnology - "chromatic technology" - might use "quark barriers", surfaces
as unbreakable as a proton. (From what I know of quantum chromodynamics, this
is incredibly unlikely. The bubble would have to be pushed out by truly
inconceivable forces; pre-Plank-time Big-Bang-type forces. But hey, what do I know?)
Then there's the simple possibility, dependent on molecular details of which I
know nothing, that defense on the molecular level has an innate advantage over
offense. Certain molecules or nano-screens might simply be easier to form
than to break, just like atoms or quarks. As long as it is theoretically
possible to pierce the screen, the possibility of piecemeal destruction
remains - unless the screen is easier to "heal" around the breach, regaining
the broken area. Or the screen might be theoretically impossible to breach,
which is what I've been speculating about all along.
For that matter, one can speculate about a zone of space being made friendly
to one force and hostile to another - maybe with electrical power being
transmitted along scrambled frequencies; a carefully targeted attack or
resource. Maybe, even after a nuke in that area, the zone would be more
quickly regained. If attack requires "opening" to an opposite attack, or
breaching a zone, then the city might regain more than it lost. But this
seems unlikely. I can't really envision a "zone" that a nuke wouldn't
destroy. The only plausible persistent zoning would be the property of being
surrounded by the good forces, rather than the bad forces. Which means that
the bad forces can peel the good guys like a potato; each time an attack
occurs, a convexly curved surface turns flat.
Nanosystems are always faced with "destruction by induction". That is, you
can always destroy one cell; therefore you can destroy the whole thing. To
defend against this, it is required that the system expand faster than the
destruction OR that it be impossible to destroy one cell.
This actually opens up a whole new vista of defense. If a city has an innate
advantage over the sea where expansion is concerned - if a compact form beats
an englobing one, perhaps due to internal bracing or some such - the city can
easily and quickly regain any area lost, even break into space and escape.
I also speculated along similar lines in two places earlier; one with healing
shields, another with the city grabbing the additional mass required to attack
rather than defend.
Speaking of escape, there's another workaround for "destruction by induction".
If you run into space, the gray goo might hurl itself after you, but it won't
have the overwhelming resources required for "destruction by induction". In
addition, the effort of destruction might simply provide the city with a
velocity boost. Certainly things in space will be different than on the ground.
--
So there are a lot of immune systems, just as there are many forms of attack.
It's just that defense being favored over attack requires too much exotic
technology; the exotic attack technologies are far more conservative than the
exotic defense technologies. The majority of strategic considerations are in
favor of attack; the only really hopeful strategic consideration is the city
being able to quickly expand into rich zones of destruction.
Ergo, it is easier to destroy than create.
--
(1) "Harvard doesn't publish science fiction." New Destinies III, Spring
1988. Edited by Jim Baen.
--
sentience@pobox.com Eliezer S. Yudkowsky
http://tezcat.com/~eliezer/singularity.html
http://tezcat.com/~eliezer/algernon.html
Disclaimer: Unless otherwise specified, I'm not telling you
everything I think I know.
This archive was generated by hypermail 2.1.5 : Fri Nov 01 2002 - 14:44:46 MST