From: Wei Dai (weidai@eskimo.com)
Date: Thu May 16 2002 - 13:06:35 MDT
On Wed, May 15, 2002 at 07:32:57PM -0700, Robert J. Bradbury wrote:
> A singleton, at least of the MBrain form cannot easily reproduce.
> It has to wait until it navigates close to sufficient resources
> to "duplicate" itself (providing the "kinship" to keep things
> safer over the long term). It takes a *long* time to navigate
> things the size of solar systems.
How do you know an SI can't create a seed and ensure that it will stay
friendly in the long run? Eliezer in _Creating Friendly AI_ claims that
even humans can do it! I'm not convinced that his particular approach
works, but I don't see a reason why it's impossible either.
> It is futile to attempt to take on anything with robust nanotech
> that has had enough time to develop berserker-bots. The only
> way I could imagine attempting a a takeover of a berserker
> prepared civilization would be if you can generate and direct
> moderately large black holes. And then the contents of the
> system isn't much good to you.
I guess you're assuming that there will always be weapons for which no
defense is possible, the equivalent of today's nuclear missiles. I think
in that case all of the MBrains will want to be in stealth mode (envelop
your star in a shell and dump waste heat into a black hole) and hide from
each other. Occasional violence will break out when one MBrain finds
another without being detected itself. It can then safely kill the other
one and take over its resources. The victim's berserker-bots won't know
where to seek revenge.
> If robust nanotech develops and people disperse through the
> solar system relatively rapidly, and we have some means
> for collectively enforcing space based property rights
> (courts everyone respects?) I don't think there is any
> competition except in the virtual realm.
If I want to convert some iron into carbon, I will still want to trade
with someone who wants to convert carbon into iron, instead of transmuting
the elements myself. So at least there will be competitive markets in raw
materials.
Perhaps all SI's will converge to the same set of technologies, so there
will be no comparative advantages in physical manufacturing, but I see no
reason to assume that. I think it's possible that there will still be
competitive markets in manufactured goods post-Singularity.
> There its much
> easier to give up your virtual property (CPU cycles) if
> someone else has a better use for them than you happen
> to have right now.
That depends on whether there's an efficient way to do general computation
with encrypted data, otherwise you're going to want physical control over
your computational substrate. Without that technology you will
leak information to your competitors when you farm out your computing
jobs.
This archive was generated by hypermail 2.1.5 : Sat Nov 02 2002 - 09:14:08 MST