From: Eliezer S. Yudkowsky (sentience@pobox.com)
Date: Sun May 14 2000 - 23:22:17 MDT
"Robert J. Bradbury" wrote:
>
> Not really, look at your own immune system, running around checking
> the state of all the cells in your body. They are usually able to
> police the neighborhood fairly effectively and only consume several
> percent of the resources available to you.
Yes, look at the immune system. The immune system has the resources of
the complete human body behind it. It uses what amounts to controlled,
directed evolution to evolve sophisticated antibodies. And yet many
infections get a hell of a lot farther than the "hot-spot" stage before
being eradicated. And people still die of infections. Complex
organisms with millions of years of evolution behind them, using
controlled evolution to create new responses instead of relying on a
fixed set, are routinely fought to a standstill or beaten by bacteria.
No, this is not a good thing.
> So diamond is 10x stronger than steel,
> that just means your missles have to have 10x larger expolosive
> yield to produce equivalent damage to a diamondoid jet vs. a steel jet.
>
> The U.S. already makes the best jets in the world, with some competition
> from the Russians. So what is the point of funding large efforts
> to make nanotech jets? Given the computational capacity that nanotech
> will allow you to put into missiles, I question whether jets are
> viable weapons *at all* in a nanotech era.
Oh, they're terribly viable weapons if *you've* got them and the enemy
*doesn't*. So, diamond is 10x stronger than steel... that means your
jet might be light enough for you to *outrun* non-diamondoid missiles.
Or launch small but incredibly fast and explosive countermissiles guided
by nanocomputers. And so on. Do you really think that nanotech
implementations of conventional weaponry would not be crushingly more
powerful than the conventional weaponry of an unarmed opponent? Such an
opponent might be, in effect, "unarmed" indeed.
> > followed by saturation launches of vat-grown nuclear weapons.
>
> The problem with nuclear weapons is you *still* need the uranium.
> Robert has told me that Gina has a copy of a paper he did that says
> "mining" gold using nanotech may be prohibitively expensive.
> Mining uranium may be even worse.
Actually, I think the last time the Extropians list went through this I
made a proposal for inexpensively creating fusion weapons. I don't
think I'll repeat it. But you get the point.
> To grow nuclear weapons you are going need (a) fuel and (b)
> rad-hardened nanotech. I seem to recall that during the
> development of the A-bomb we lost at least one scientist
> at Los Alamos because he picked up a hunk of plutonium to
> prevent an accident. What makes you think nanotech can
> work in that environment? Nuclear fission works by
> the release of neutrons, a kg of fissionable material
> releases a lot of neutrons and doesn't matter *what*
> the nearby material is, it is going to sustain damage.
If flesh and steel can construct nuclear weapons, so can nanotech. If
nanotech can withstand solar radiation for a year, it can withstand
uranium for an hour. If nanotech can function with sufficient
non-fragility to be debugged, the finished product can function despite
substantial amounts of radiation damage.
I mean, okay, "rad-hardened" nanotech. Why is this at all difficult?
Getting from sol-hardened to nuke-hardened strikes me as being a lot
less difficult than getting from zero to sol-hardened.
> > We'd have to get through that without blowing ourselves up before
> > we'd get the chance to dust the biosphere.
>
> We have enough nuclear and bioweapons today to blow ourselves up
> and we aren't doing it.
Yes. Why?
1) MAD works when exactly two (2) sides have the weapons. (It's a good
thing that the Evil Empire has disintegrated, leaving thousands of
nuclear weapons in the totally trustworthy hands of a dozen bankrupt and
corrupt ministates. It makes me feel so much safer.)
2) MAD also requires time to spot the weapons coming in and launch a counterstrike.
3) Nuclear weapons aren't self-reproducing. Getting a thousand nuclear
weapons requires a lot more effort than getting one nuclear weapon,
giving the opponent time to catch up and maintain the balance of terror.
4) There is no economic benefit, no positive side, to nuclear weapons.
Nuclear weapons can destroy a city, but not conquer it.
5) The last time there was a real war on this planet, the United States
developed ultraweapons before anyone else, in secret, and used them
against a non-nuclear enemy.
> You *MUST* make a credible situation
> that someone will (a) get nanotech before anyone else; (b)
> use it to develop advanced weapons; (c) grow such large quantities
> of those weapons and train people to operate them (or create
> "intelligent" weapons) that they represent a threat; (d) do it all
> without anyone realizing what is going on.
In the simplest case, I only have to argue that Moscow will *think* such
a situation is plausible and threaten to launch a first strike unless
the US gives up nanotechnology development...
Heck, right now, Moscow is screaming because the US is talking about an
ABM defense.
> Then after you have
> met *all* of those critera, you have to present a case that
> the threat would be so large and present itself so fast that
> conventional weapons and tactics would be useless against it.
Conventional weapons and tactics would be useless against a rain of the
thousand nuclear missiles we have now, never mind ten thousand
diamondoid ones.
> > Even if Freitas's paper is entirely true in every
> > detail, it doesn't mean - as a policy conclusion - that there's a safe
> > path to nanotechnology.
>
> But it is a first attempt to show that all of the "crying wolf"
> that people do *may* be crying at shadows on the wall rather than
> real wolves.
If so, I am not convinced. I regard Freitas's paper as a useful
delineation of some mathematics and battle tactics of nanotechnological
warfare. I saw nothing in it, at any point, that made me feel safe.
Anyone who gets a warm fuzzy feeling from reading Freitas's paper is
deluding verself as severely as Neville Chamberlain.
> You can't *prove* that we aren't going to be turned
> into dust tomorrow by an asteroid we didn't see. All you can do
> is make reasonable estimates that its highly unlikely to occur.
> That is about the best we can do with nanotech as well.
>
> > (2) The battle strategies depicted here aren't twisted enough to depict
> > a real-life arms race. Would military replibots really wait around
> > passively to be swept up into neat little nets?
>
> No, but if you are busy avoiding being swept up that is likely to
> put a significant restriction on reproductive efforts. The nets
> could simple herd the bad-bots into an area where they exhaust the
> resources and as a result are forced to stop replicating.
The bad-bots don't cut through the nets? Eat the nets? Sweep the
goodbots into nets? Change their external configuration to look like
goodbots? And so on. I'm not the one to run this argument, because I'm
not a nanotechnologist. Right now, it does look to me like there's a
distinct tendency on the part of the "nanodoves" to assume that, once
you have presented a means whereby the defense can overcome offense,
you've won the argument. You don't go on to ask: "How would I go on to
outwit this defense?" Presenting a scenario in which defensive nets
sweep up the bad guys is not an argument.
I mean, for accidental grey goo, it's a great argument. Don't get me
wrong. But for military-grade nanotech, this is rather like saying that
tanks will never be important on the battlefield because the defensive
tanks could take a big net and sweep up all the offensive tanks into a
neat little bag. You can always give an example of defense overpowering
offense. I can always give an example of offense overpowering defense.
What matters are the overall technological forces determining who wins.
I see three major forces:
1) Offensive bots need to reproduce. Defensive bots don't need to
reproduce, and assuming early detection, will always outnumber the
offensive bots.
2) Defending against an attack requires considerably greater technical
sophistication than the attack itself.
3) An offender can use scorched-earth strategies that are closed to the defender.
> > [snip] and can scan or alter every byte of RAM; are we really supposed
> > to win a cracker's war in physical reality?
>
> It depends how important it is to you. We don't have good computer
> security now because it is too much trouble for humans to remember
> 3 passwords. However we do have retina and fingerprint recognition
> devices coming that will make it *very* hard to gain improper access.
Have you ever read anything from Cult of the Dead Cow? Actually, you
probably have. There's one fascinating article that describes how you
can take over a computer if someone is copying a C string to memory
allocated on the stack without checking the C string's length; you
overwrite the stack's return address to go to a place in ROM that will
execute the rest of the program that you sent in the C string.
(Actually, because the C string is zero-terminated, you can't use a zero
byte in the program, so you have to send the program as a
ones-compliment version, then send the return address to a place in the
ROM that will take the ones-compliment or subtract 0x80 or whatever and
*then* execute the program.)
No password required. No retina required. I wasn't talking about
attacks against the password. I was talking about the kind of deeply
codic attacks invented by crackers who might call themselves "hackers"
and be worthy of the name.
> The *reason* most medical procedures are so expensive is that
> you have to do almost everything possible to avoid killing the
> patient. Most computer systems can't "kill" their users and so
> security failures are tolerated.
So what you're saying is that perfect security is achievable, we just
aren't trying hard enough? I do not believe this to be the case.
> > (3) Assuming that a single radiation strike produces device failure is
> > conservative when arguing a proof-of-possibility that nanomachines can
> > be constructed.
>
> I can solve this very simply. Researchers at Berkeley discuss
> a high energy cesium ion source that can deliver 0.4 Amps of
> ions in 0.37 microsecond pulses (delivering 400 trillion watts).
> Any nanotech in this beam is converted to *slag*. If you think
> nanotech has active electrostatic or magnetic defenses, you just
> convert the beam to neutrons as is done in the Berkeley
> Rotating Neutron Source.
No, I was objecting to Robert Freitas using the one-point-of-failure
assumption to argue that aerovores would require large opaque shields
(which would slow down the reproduction rate). I wasn't saying that
aerovores would be immune to energy weapons.
> > Why call attention to ourselves now, risking a media brouhaha and a
> > ban on all nanotechnology?
>
> A ban will not work.
In the long run, no. But it could very easily delay nanotech
development in the US until it's too late.
> You can't regulate small technologies that
> can be constructed from relatively abundant materials.
You can throw Jim von Ehr in jail.
> How do
> you ban AFM construction? Ok, lets regulate the sale of all
> peizoelectric crystals. Hmmm, doesn't that mean I have to ban
> the sale of all elements from which those crystals can be grown?
> Right!
Let me get this straight: You're saying that the good guys can detect
particles the size of a microbe in every bird, bee, and mudhole on the
planet, but our modern government can't find a back-alley peizoelectric
lab? Okay. I can maybe accept this. But I don't buy illegal
back-alley nanotechnology startups and covert NSF grants. If the
government shut down all currently operating labs and put all current
nanotechnology researchers and Foresight members and sci.nanotech
posters under surveilance, nanotechnology in the US would be effectively over.
> > By the time global ecophagy is a real threat, everyone will have been
> > screaming about diamondoid fighter jets for the last year, and there'll
> > be moratoriums all over the place.
>
> Why? If the nations of the free world see this coming, the probability
> will be that they all will have diamondoid fighter jets within a few
> years of each other.
"A few years?" A heck of a lot can happen in a few years! If the USSR
had developed nukes "a few years" before the US, instead of vice versa,
there would have been hell to pay.
> You don't think the EU or China or Russia is
> going to sit by twidling their thumbs while the U.S. develops
> this technology do you? Technology is technology, the economic
> advantages benefits of the technology used "properly" are very
> clear, so the development is likely to occur at relatively similar
> rates around the world (in the more developed nations).
Can you give me *any* historic incidence of anything *remotely* like
this scenario developing? Even taking nuclear weapons as the model
might be too conservative.
> > A non-governmental-organization (i.e. Foresight) publishing
> > voluntary guidelines is good enough for the immediate memetic effect,
> > too; you just say, in a heavy, serious tone of voice: "The Foresight
> > Guidelines explicitly prohibit the development of replicators which can
> > operate in a biological environment; furthermore, the Foresight
> > Guidelines require that even vat replicators use a broadcast
> > architecture..." and so on.
>
> You want to go further than that, you want to make people aware that
> it is a serious problem,
Why? What's the value-added for Them becoming aware that military-grade
nano is a serious problem *now*, rather than later (but still before
it's an actual problem)?
Do you think the government would have allowed the development of the
Internet if the MPAA (or whatever the music industry calls itself) had
known about MP3s twenty years in advance? If Barnes and Noble had known
about Amazon? If Clinton had known about the McCain campaign? If the
saddle industry had known about Ford? Progress only happens because the
government and special interest groups don't become aware of it until
it's too late.
> but one that can be defended against with
> a safety level that corresponds to your investment in sensors and
> defensive devices. Wouldn't it be better to put into place (legally,
> on a global basis) things like the NIH guidelines that work to prevent
> accidents or the bio/chemical weapons treaties that managed to begin
> putting the bad-genies back into the bottle shortly after it got out.
It would be a good idea to have NIHoid guidelines in place *after* the
civilian benefits of nanotech become clear, *after* there are special
interest groups *defending* nanotech, but *before* ecophages became possible.
--
sentience@pobox.com Eliezer S. Yudkowsky
http://pobox.com/~sentience/beyond.html
Member, Extropy Institute
Senior Associate, Foresight Institute
This archive was generated by hypermail 2.1.5 : Fri Nov 01 2002 - 15:28:37 MST