From: Charlie Stross (charlie@antipope.org)
Date: Tue Jan 18 2000 - 07:10:18 MST
On Tue, Jan 18, 2000 at 02:51:32PM -0500, Robert Bradbury wrote:
>
> Now the problem is *not* having an assembler in your home. The
> problem is in using it dangerously or incorrectly (sounds like
> the radio-transmitter/wiretapping debate). So really the problem
> comes down to how do you prevent general purpose assemblers from
> being used incorrectly. Well it sounds like we are back not to
> regulating the assemblers but regulating the *designs*!
If I was going to do this, I would use the Ken Thompson hack.
A Von Neumann machine is a compiler; it takes designs in at one end
and translates them into a different form -- in this case, physical
matter.
Ken Thompson -- inventor of UNIX -- did a very interesting hack in
the early days, which effectively gave him root access to any UNIX
system. The hack was self-propagating (it would migrate to new
architectures when you ported UNIX to them), and was not visible in
the source code. You can find his seminal paper on it ("Reflections
on trusting trust") at:
http://www.cs.umsl.edu/~sanjiv/sys_sec/security/thompson/hack.html
The Thompson hack works like this:
Say there is a program you want to compromise, which is created
by a compiler. (An example: the UNIX login program invoked whenever
a new user logs in. Purpose of corruption: to make it recognize
a special password and give whoever enters it carte blanche on
the system.) You hack the _compiler_ so that when it recognizes
the source code for the login program, it inserts some extra code
before compiling it. Thus, the source code for the login program is
clean.
But there's more. You could in principle detect this hack by examining
the source code for the compiler, right? Wrong. Because you apply
the same hack to the compiler; you prepare source for a "corrupt"
compiler (which inserts the back door into the login program), and
modify it so that it _also_ inserts an equivalent back door into the
compiler -- one that detects compiler source code and inserts the
back door. You then distribute a corrupted compiler binary, and
clean source code for the compiler. When the unsuspecting user
compiles the compiler against a new machine architecture, the output
will propagate the back door ...
A similar hack can in principle be applied to Von Neumann machines (such
as a Drexlerian household assembler). You ensure that if the machine
is told to create certain items (berylium hemispheres, for example --
tamber-pieces for nuclear weapons -- it goes ahead but also silently
builds a cellular phone (or equivalent) and snitches to the nuclear
weapons inspectorate.
To this end, you'd load any commercially available assembler with a
library of "danger" signs to watch out for. Tamber pieces, botulinum
toxin, precursors for chemical weapons, that sort of thing. You'd
also ensure that its design for replicating itself files some sort of
registration with you whenever a user tells it to reproduce -- and this
includes components of a Von Neumann machine that can be assembled
later.
This hack doesn't prevent an assembler being used for nefarious
purposes. However, it stops it being used *stupidly* for nefarious
purposes -- if you want to use a domestic assembler to declare war on
your neighbours, you either have to invent a wholly new weapon that
won't set off the built-in snitch circuits, or you need to build your
own "clean" assembler *from scratch*, without using components from a
commercial assembler that may be contaminated by police Thompson hacks.
Both of these are significantly more difficult operations than simply
running off a couple of litres of Sarin.
For an added twist, we could make the police hack a meta-hack;
instead of looking for one suspect behaviour, and replicating itself
into any derived assemblers, the hack might check a public bulletin
board for a whole range of suspect design items, and build additional
recognizers into itself for each one. So unless you grow your assembler
farm in a firewalled asteroid, it's going to be brought up to date on
no-no forms of behaviour every time you add a node to it. (I'm assuming
that the exponential growth capabilities of a Von Neumann machine will
be of interest to anyone planning mega-havoc with one.)
Make looking for grey goop components an open source operation; you
get kudos for coming up with a design template for something nasty
*and submitting it to the police site*, from which new assemblers
will grab it and configure themselves to snitch on anyone who tries
to make one. "Extra brownie points for submitting that alternate
klystron design that those terrorists tried fabbing last week! Thanks
to you, they didn't get their bomb finished before the police turned
up with a search warrant. _And_ you made your neighbourhood safer."
> I think the general direction will be "non-tamperable" assembler
> boxes (i.e. if you open it, it turns into smelly sticky gloop)
> and encrypted (and verified) designs. So while the design
> itself is open source, it has to be verified, check-summed,
> encrypted and transmitted to the assembler by "approved"
> organizations. This is similar to the way that DVD's were
> supposed to work (until someone messed up and forgot to encrypt
> the decoding keys in one vendor's product...). The assemblers
> (similar to the DVD players) will only assemble designs that is
> approved and untampered with.
I think this is too difficult. It suffers from the same problem as
Microsoft's ActiveX components; anyone can write one, obtain a
certificate, certify it as their own, and distribute it. If you can
corrupt the certificate issuing authority and get your hands on a
certificate, you can certify your design as 'clean' and wreak havoc
with it. Thus, there's a single point of failure. Who will guard
the guardians?
:
> Now, the real trick will be creating some separation of power
> between organizations, such as "governments" (who might want to
> control behaviors) or industry "monopolists" (such as the DVD
> consortium) and the "verification/approval/encryption" organizations.
> Presumably you would want something like this done by some kind
> of non-profit foundation whose primary interest is in verifying
> that the designs are safe, not in regulating what people do with
> them. You do not want to get into a situation where someone like
> Natasha designs something but since it doesn't fit the paradigm
> of how the "powers" want the society to look, she can't get the
> approvals required to get it assembled. I suspect this will
> end up looking a lot like the current drug approval processes
> (with a lot more engineering science).
No.
No organisations. No licenses.
Instead, make it a bottom-up process that is in the interests of
sane individuals to contribute to. Take the best of the open source
movement. People support and contribute to it because they get
something back. Torvald's law applies: "given an infinite number of
eyeballs, all bugs are shallow". In this case, given an infinite
number of eyeballs, all possible attempts to trick an assembler
into building something really nasty can be anticipated and blocked.
> I believe in the long run (20-30 years), if we get Minting and we
> get some reasonable IA for design, you will get just about "anything"
> boxes. The tough part may be getting some rare element (e.g.
> gandolinium(?) for magnetic refrigerators) so there may still
> be some interesting supply constraints. But as always there
> will be alternate approaches and substitutions so these may
> not be strongly constraining.
I'm waiting for the Free Hardware Foundation -- the post-nanotech
equivalent of the FSF. Same motives, same methods, same results.
Why buy a Ford when you can customise your own GNU Automobile? Sure
it won't be as stylish unless you make the effort to style it yourself,
but it'll be a hell of a lot cheaper, and it'll be reliable, too. (I
expect it will have all the aesthetic charm of a Humvee, with added
solar power/coal/nuclear propulsion options -- but you have to paint
it yourself.)
The big thing we can anticipate from assemblers is the commoditization
of physical goods that are currently premium items. A lot of things we
currently spend money on and invest with a sense of value will drop off
the map, insofar as they will stop being expensive or difficult to
make. Think of the mutation of the book from mediaeval illuminated
manuscript to battered paperback. This doesn't mean we don't have books
any more -- we have many more than ever before -- but they don't cost
six months' income or require hand-crafting on an individual basis.
We already expect the cost of consumer electronics to fall, month on
month; the same effect will hit other types of physical gadget once
the nanotech revolution begins.
By 2030, we'll be thinking of a middle-class 1990's existence -- in
material terms -- as being poverty-stricken. Because the real defining
items of wealth will have moved away from manufactured goods; the
emphasis will be on handicrafts, bespoke designs, and information.
-- Charlie
This archive was generated by hypermail 2.1.5 : Fri Nov 01 2002 - 15:26:18 MST