From: Ben Goertzel (ben@goertzel.org)
Date: Sat Jan 10 2004 - 06:48:36 MST
I wrote:
> Well, my views have changed a bit. I am now reasonably intuitively
> confident that the revised Novamente design will lead to a Friendly AI.
I should have elaborated on this a bit. The Novamente AI design has been
revised in a way that, among other consequences, appears to make it
compatible with a goal system similar to the one you've described in CFAI.
The concerns I have repeatedly expressed before, regarding the stability of
a Friendly goal structure under repeated self-modification, still exist.
Intuitively, under the modified Novamente design, it seems that these
concerns are not likely to cause a problem. However, one would like to have
more than an intuition on such a matter, of course!
I actually think I see the direction one must go in, theoretically, to make
a rigorous analysis of self-modifiable-goal- structure stability. However,
the math will be very hard, and I am not going to be focusing time on this
in the near future. It may be that semi-AGI-based theorem-proving software
(which I think I know how to make using Novamente, though it's a large
project unto itself) will be useful for exploring this sort of theory.
-- Ben G
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:44 MDT