From: Daniel Burfoot (daniel.burfoot@gmail.com)
Date: Sun Nov 25 2007 - 19:39:53 MST
On Nov 26, 2007 10:37 AM, Rolf Nelson <rolf.h.d.nelson@gmail.com> wrote:
> If I granted your premise, and thought I lived in a country where the
> government is fundamentally tyrannical, then I would agree that it would be
Several problems with this line of reasoning:
1) Gov. could be fine today, but devolve into tyranny tomorrow. This
happens frequently in historical terms.
2) Short term applications of AI can be used to consolidate power in
the hands of a few, making it easier for them to establish a
dictatorship.
3) Research done in putatively non-dictatorial country A could be used
to enforce dictatorship in country B.
Note that item 2) is qualitatively different from the obvious analogy
to nuclear weapons. Nukes are powerful tools in strategic standoffs
between countries, but cannot really be used to consolidate power
within a country.
An interesting analogy exists between FAI theory and libertarian
theory. In both cases, one considers the design of an entity that is
vastly more powerful than any individual. Naive people think it will
be easy to control the {AGI, state} to make our lives better. {FAI
researchers, libertarians} think this is pants-on-head crazy, and want
to build strong safeguards into the design of the {AGI, state} to make
sure it doesn't kill us all.
Dan
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:01:01 MDT