From: Billy Brown (bbrown@conemsco.com)
Date: Mon Mar 29 1999 - 09:39:05 MST
Nick Bostrom wrote:
> Billy Brown wrote:
> >I won't pretend to know what it will actually do at that point,
> > but I can't see it being concerned about something as prosaic as its
> > supply of atoms.
>
> Why not? If it is better off (however slightly) with these atoms than
> without them, then in this scenario could all be dead, unless we have
> been wise enough to make sure that the power is ethical..
Note: I assume here that we're talking about strong SI and some variant of
the rapid self-enhancement scenario.
Well, first off, I think an SI is going to invent its own ethics long before
it really deserves that label. IMO, everything any human has ever thought
of on the topic is going to seem intuitively obvious to such an entity.
Now, whether its morals will agree with ours is still an open question
(although I side with Eliezer in expecting that the SI is more likely to be
correct than we are).
But that aside, atoms are mostly useful for bulky, clumsy, primitive
applications like nanotechnology and electronic computers. An SI with
moderately advanced nanotech can convert a few cubic meters of mass into a
computer in hours or less, and would probably do so. However, the delays
involved in manipulating bulk matter on any larger scale are going to seem
agonizingly long once it ports itself to that first machine. How willing
are you to wait several days to complete the project if your consciousness
runs 10^12 times faster than that of a human?
IMHO, what happens next depends on the ultimate nature of physics. I see
two important cases:
If it is possible to migrate to some kind of custom-engineered reality, the
SI will do so before it has existed long enough to bother with changing any
significant portion of the Earth (moving atoms is so *slow*, after all).
There is a very small chance that this could have side effects that would
destroy Earth, but it seems unlikely - if such a project is not possible at
low energy levels, the SI will probably have to migrate to some location
where high energy densities can be achieved.
If such projects are not possible, the SI will probably migrate to whatever
part of the universe is best suited for its projects. Since all of the
options that have been suggested require exotic physical conditions (i.e.
the surface of a neutron star), I conclude that there is little chance it
would want to stay here. So, once again the odds are in our favor.
Billy Brown, MCSE+I
bbrown@conemsco.com
This archive was generated by hypermail 2.1.5 : Fri Nov 01 2002 - 15:03:25 MST