From: Ben Goertzel (ben@intelligenesis.net)
Date: Mon Dec 11 2000 - 07:13:10 MST
> This all gets complex. You'd have to read "Friendly AI" when it comes
> out.
I do intend to ;>
> But the first steps, I think, are: (1), allow for the presence of
> probabilistic reasoning about goal system content - probabilistic
> supergoals, not just probabilistic subgoals that are the consequence of
> certain supergoals plus probabilistic models.
This is a key point, and really gets at why your "I love mommy and daddy, so
I'll
clone them" example seems weird.
We try to teach our children to adopt our value systems. But our explicit
teachings
in this regard generally are LESS useful than our concrete examples.
Children adopt
goals and values from their parents in all kinds of obvious and subtle ways,
which
come from emotionally-charged interaction in a shared environment.
Choice of new goals is not a rational thing: rationality is a tool for
achieving goals, and
dividing goals into subgoals, but not for replacing one's goals with
supergoals....
Perhaps this is one of the key values of "emotion." It causes us to replace
our goals
with supergoals, by latching onto the goals of our parents and others around
us.
>(2), make sure the very
> youngest AI capable of self-modification has that simple little reflex
> that leads it to rewrite itself on request, and then be ready to *grow*
> that reflex.
Rewriting itself on request is only useful if the system has a strong
understanding of HOW
to rewrite itself...
ben
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:35 MDT