From: Anders Sandberg (asa@nada.kth.se)
Date: Mon Aug 25 1997 - 18:02:26 MDT
"Eliezer S. Yudkowsky" <sentience@pobox.com> writes:
> Cycles:
> 1A. Programmer writes a piece of code to do something.
> 2A. Code evolves into something small, dense and tight.
> 1B. Code leaps into new evolutionary space and discovers new algorithm.
> 2B. Programmer expands algorithm and generalizes it.
Sorry, step 2A doesn't occur that easily. GAs tend to produce
bloated, redundant and weird code; their strength is that they are so
good at finding flexible solutions.
The basic idea above is sound, Thomas Ray has suggested something like this
to get completely unexpected solutions to problems nobody has yet posed.
> > It is this assumption I want to challenge. If it has the tremendous
> > destructive potential you assume, it is a fairly logical assumption.
> > But can you really back it up with some hard calculations?
>
> How much time would it take for a nanomachine to construct a nuclear weapon?
> I think we can assume that nano is at least as destructive as nuke.
Numbers, please. It is easy to claim something like this, but is it
really true?
Building a nuke: you need around 10 kilograms of uranium 235 (or whatever
isotope it was). There is around 2 grams uranium / tonne in the crust of the
earth, of which 0.72% is U235, so to get 10 kg you need to process around
7000 tonnes of crust. I'm not sure how much energy is required to reduce
the UO2 to pure U, but it is a noticeable amount (are there a chemist
in the house?). Assuming the nanites cover a large patch with solar
collectors, they can get around 500 W/m^2, which has to cover their
replication, search through the crust, reduction, isotope separation and return
to the "base"; how much energy this is is a bit hard to tell right now
(it is 1.30 in the morning here :-), but it looks like it will take
a while for the bomb-mold to blow up. A wild guess would be around a
week.
Doing the same work as a nuke with nanites (i.e. disassembling everything
within a few hundred meters and blasting everything within a few kilometers)
is rather tricky, since it is extremely energy intensive. You need plenty
of energy to do the disassembly (essentially you have to break most molecular
bonds), and nanites are bad at making blast waves. OK, this is an unfair
comparision since nanites are more similar to bacterial or viral infections.
I would like to do some more careful calculations on this (or rather, that
somebody with the right knowledge did it instead of a bumbling amateur like
me), but it is IMHO clear that we should not be overly worried about
nano-built nuclear weapons but rather nano-ebola. Everything in the world
has limits set by the laws of nature.
> Remember that the Bad Guys aren't operating under the same restrictions as the
> Good Guys. The wannabe dictators will use directed evolution on a scale no
> Good Guy would dare to contemplate. And evolving mutually competing predators
> will go a lot faster than mutually supporting immune systems.
Ever thought about the fact that the immune system isn't mutually supporting?
It is rather a balance of predators which do not predate on each other since
that destroys their fitness.
Good guys will be motivated to contemplate directed evolution since the bad guys
do it; and they tend to have more brainpower and money on their side. I
think the weapon of openness is important here.
> Our immune systems are unimaginably more sophisticated than a virus or a
> bacterium, using controlled evolution to combat natural evolution. And yet we
> still suffer from colds and diseases. The only reason that the viruses
> haven't killed us outright is that it isn't good strategy.
Exactly. So the major question is: is it possible to create a nanite
infection that is deadly (or subtle) enough to wipe out all competition?
Don't reflexively answer 'yes' to it, try to give a considered answer
of why it is likely (or why not). Note that it would have to be able
to spread across the world fast enough to subvert everyone, or circumvent
all attempts at detection. That is assuming a lot.
> I want to repeat this, because it's important. Our immune systems are the
> closest analogue to proposed nano-immunities. The mismatch in available power
> and sophistication is enormous. Our immune systems learn from experience, use
> controlled and directed evolution, have memories... everything but the ability
> to consciously design things. And yet the viruses waltz casually through our
> bodies, because it's so much easier to destroy than create.
Have you studied medicine? Then you know viruses certainly do not waltz causally
through your body (of which around *one kilogram* apparently is immune cells, if
you are a normally built male).
> It is easier to destroy than create!
You repeat this as a mantra. And of course you have the second law of
thermodynamics on your side. The problem is that you do not attempt to
make quantitative comparisions between the strengths of different systems,
and instead rely on plausible-sounding arguments. That is definitely
*not* a good strategy if you are trying to discuss something important
where we do need a well planned policy.
> If any human is even capable of designing an immune system,
> then the average educated person will be capable of breaking it, given time
> and effort. Any twisted genius will go through it like tissue paper.
Sigh, here we go again. What is your evidence for this? A few months ago
somebody on sci.nanotech posted a challenge for people to come up with
nanotech immune systems and he would find flaws in them. I don't think
there was any follow-up, which is sad, but it might be worth taking up
here on the list. Unfortunately I do lack the time to deal with this
as carefully I would like, since I consider this very important, but
let's try a simple sketch to see how easy it is to vanquish a designed
immune system:
The body is surrounded by an inert skin (say diamond); attempts to
physically breach it can be detected from the inside and the surrounding
region sealed. The rest of the organism (could be a transhuman, factory
or a city) is compartmentalized with similar walls; infected sections
can be isolated. Immune devices move around, interrogating "cells"
(subsystems) by comparing their surface markings with allowed types
(this list can be kept secret from someone who disassembles a device
by using a trapdoor function), and occasionally disassembling the
cell to check its innards. Other immune devices check the general
activity, looking for deviations from the normal state (there is
a paper on the net somewhere about using GAs to evolve anti-hacking
agents, this is something similar). If a compartment is found to be
infected it is terminated with extreme prejudice (the human body
uses the granulocytes and their nasty cocktail of free radicals,
chlorine acids and proteolytic enzymes, this would use something
worse or simply throw out the infected compartment from the
organism). Note that production of new immune devices can be done
by assemblers in well defended deep regions, and that the
devices themselves can be fairly simple since they do not need
to replicate, just find threats.
So, what are the obvious holes?
-- ----------------------------------------------------------------------- Anders Sandberg Towards Ascension! asa@nada.kth.se http://www.nada.kth.se/~nv91-asa/main.html GCS/M/S/O d++ -p+ c++++ !l u+ e++ m++ s+/+ n--- h+/* f+ g+ w++ t+ r+ !y
This archive was generated by hypermail 2.1.5 : Fri Nov 01 2002 - 14:44:46 MST