From: hal@finney.org
Date: Fri Jun 08 2001 - 01:05:27 MDT
Eliezer writes:
> Okay, by request, definitions. See also "Creating Friendly AI" (of
> course).
>
> "Child goal": A relation between two goals. "C is a child goal of B".
> "B is a child goal of A".
>
> "Parent goal": A relation between two goals. "B is a parent goal of C".
> "A is a parent goal of B."
>
> "Supergoal": A category of cognitive object. The set of events that a
> given mind views as having inherent desirability.
>
> "Subgoal": A category of cognitive object. The set of all events that
> are desirable only as a means to an end.
Aren't you giving idiosyncratic meaning to relatively widely used
terms within AI? Supergoal and subgoal traditionally mean roughly what
you are here calling parent and child goals. I think it is going to
foster confusion to take existing terms and give them your own peculiar
definitions.
Philosophically, it's not clear that the distinction you have drawn
between super- and subgoals is well defined or even meaningful.
One problem is that you describe them as being relative to a given mind.
But goals are a property of systems without minds as well. Animals,
even very primitive ones, plants, bacteria can all be said to have goals.
Do you mean to claim that the goal structure for mindless creatures does
not have super and subgoals? Why is it that when something evolves a
mind, suddenly some goals have "inherent" desirability and others are
only a means to an end?
Another problem is that it's not clear that there are any supergoals
as you describe them, or at least that there are any beyond the trivial
goals of survival and propagation of genes (which, again, apply to lower
life forms just as much as to humans). Ultimately everything comes down
to that, we just have different strategies to achieve these goals.
There is also the issue that we are not always consciously aware of our
true goal structure. We think we are doing something for one reason
when the actual purpose is something else. Look at the results in
sociobiology, like Robin's recent article on why we believe even the
poor should have good health care. If these analyses are right then
what we believe about our goals is not very accurate.
If we don't know what our true goal hierarchy is, then we may be
completely mistaken about whether something is a subgoal or a supergoal
in your sense. We may think that a given goal has inherent desirability
while we are actually following that goal in order to attract members
of the opposite sex. Does that make it a supergoal or not?
I think if you stick to the more traditional notion of supergoals and
subgoals that you can still achieve many of your rhetorical aims. It is
still valid to criticize an argument which supposes that a subgoal will
inherently override a supergoal. You don't have to give supergoals this
exalted and somewhat mystical state of inherent validity, they just have
to form the justification and purpose of the subgoal.
Hal
This archive was generated by hypermail 2.1.5 : Sat Nov 02 2002 - 08:08:01 MST