From: Eugen Leitl (eugen@leitl.org)
Date: Wed Jun 26 2002 - 02:56:44 MDT
On Wed, 26 Jun 2002, Stephen Reed wrote:
> I understand from CFAI that one grounds the concept of Friendliness in
> external referents - that the Seed AI attempts to model with
> increasing fidelity. So the evolving Seed AI becomes more friendly as
> it reads more, experiments more and discovers more about what
> friendliness actually is. For Cyc, friendliness would not be an
> implementation term (e.g. some piece of code that can be replaced),
> but be a rich symbolic representation of something in the real world
> to be sensed directly or indirectly.
Thanks for offering digests/interpretations of CFAI for those of us who're
too busy. Much appreciated.
Due to extreme power difference a SI can't help coupling to the ad hoc
consensus ethics encoded in external referents, warping it over the course
of a few iterations. If it eventually detaches from the referents'
floating database the referents' morality will drift away from the
now-fixed SI morality metric.
Assuming, it doesn't start with a grave misunderstanding outright.
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:39 MDT