From: Stephen Reed (reed@cyc.com)
Date: Wed Jun 26 2002 - 00:19:32 MDT
On Tue, 25 Jun 2002, Ben Goertzel wrote:
> The hard part is: If one creates a system that is able to change its concept
> of Friendliness over time, and is able to change the way it governs its
> behavior based on "goals" over time, then how does one guarantee (with high
> probability) that Friendliness (in the designer's sense) persists through
> these changes.
I understand from CFAI that one grounds the concept of Friendliness in
external referents - that the Seed AI attempts to model with increasing
fidelity. So the evolving Seed AI becomes more friendly as it reads more,
experiments more and discovers more about what friendliness actually is.
For Cyc, friendliness would not be an implementation term (e.g. some piece
of code that can be replaced), but be a rich symbolic representation of
something in the real world to be sensed directly or indirectly.
So I regard the issue as one of properly educating the Seed AI as to what
constitutes unfriendly behavior and why not to do it - via external
referents.
Perhaps my feeling of confidence regarding goal-drift stems from
experience with symbolic logic knowledge representation, and deductive
inference with justifications. I would have less confidence if my
system's goals were implemented entirely with (say) neural nets whose
explanations and boundry behavior may be opaque.
> So the major flaw I see in the Friendly Ai concept -- lack of persistence of
> Friendliness thru self-modifications -- probably won't be a worry in Cyc,
> for the same reasons that I think Cyc will never be a real AGI in anything
> near its current form:
Agreed. We will try to evolve Cyc with additional intelligent behaviors
as sponsored opportunities arise and/or open-source developers add them.
-Steve
--
===========================================================
Stephen L. Reed phone: 512.342.4036
Cycorp, Suite 100 fax: 512.342.4040
3721 Executive Center Drive email: reed@cyc.com
Austin, TX 78731 web: http://www.cyc.com
download OpenCyc at http://www.opencyc.org
===========================================================
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:39 MDT