From: Eliezer S. Yudkowsky (sentience@pobox.com)
Date: Fri Apr 05 2002 - 13:33:02 MST
Personally, I would tend to worry more about the effects of evolution on the
moral architecture. Currently we have, as the emergent result of some
political adaptations plus some new cultural knowledge that was not
environmentally present, the ability to look back at evolution and be
horrified. We have the ability to use contraceptives. If, for some
unimaginable reason, evolution continued for another million years, I'd be
worried that we'd end up as Moties - organisms with intelligence almost
completely enslaved to reproductive drives. Right now we're operating on
momentum; intelligence is a recent innovation to which the rest of the
cognitive architecture has not yet had time to fully adapt, and so
intelligence plus the rest of the architecture yields behaviors that are
interesting and beautiful from our perspective but of no utility or negative
utility to evolution. Just the fact that we can distinguish our perspective
from evolution's is one such emergent behavior. Under natural evolution,
I'd worry that we ourselves would lose the complex structure of moral
reasoning that I'd like to transfer to Friendly AIs, and end up as bacterial
intelligences with the single top-level goal of infinite reproduction and no
philosophical complexity.
Of course, evolution is about to be slapped into oblivion, so who cares.
-- -- -- -- --
Eliezer S. Yudkowsky http://singinst.org/
Research Fellow, Singularity Institute for Artificial Intelligence
This archive was generated by hypermail 2.1.5 : Sat Nov 02 2002 - 09:13:16 MST