From: Gordon Worley (redbird@rbisland.cx)
Date: Wed Mar 21 2001 - 15:22:56 MST
At 3:10 PM -0500 3/21/01, Eliezer S. Yudkowsky wrote:
> > Such a situation would be dangerous because an un Friendly SI could
>> slip thorough and do something horrible, because all that there would
>> be to stop ver are Friendly SIs who wouldn't hurt a fly in VR. It is
>> going to take un Friendly SIs to stop un Friendly SIs who manage to
>> escape the control of a Sysop.
>
>This seems to me like a totally wacky idea. Friendly SIs should be just
>as good at chess, or battle tactics, as unFriendly ones. A Friendly SI
>has no evolutionary history of going to war for the wrong reasons; ve has
>no need to practice nonviolence when battle strategy is required. I'd
>expect a Friendly SI and an unFriendly SI to look equally inedible to an
>external predator.
They may have the same ability, but will they act on it? A Friendly
SI is concerned not only with its own existance, but with the
existance of others. An un Friendly SI, though, is completely
selfish, but those wouldn't be a threat. It's the Evil SIs that are
greedy that we have to watch out for. The greedy ones will do
anything for all our base to belong to them, even if that means
killing off all other SIs to have all the computornium in the
universe. How could a Friendly SI stop an attack like that? It
would require an equally destructive amount of force, that could
result in everyone in the universe, or at least within a few hundred
thousand light years, dying if the greedy SI dodges it. Of course,
now I'm trying to write what a Friendly SI would think, which I can't
do, so I'll stop and agree to wait until after the Singularity to
make a final decision on how well Friendly SIs can fight un Friendly
SIs.
-- Gordon Worley http://www.rbisland.cx/ mailto:redbird@rbisland.cx PGP Fingerprint: C462 FA84 B811 3501 9010 20D2 6EF3 77F7 BBD3 B003
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:36 MDT