Re: ethics is knowable

From: Eliezer S. Yudkowsky (sentience@pobox.com)
Date: Thu Jun 07 2001 - 19:53:11 MDT


Max More wrote:
>
> At 10:50 AM 6/7/01, you wrote:
> >Would you people please, please, PLEASE get into the habit of
> >distinguishing between subgoals and supergoals?
> >
> >Subgoal content: Events or states whose desirability is contingent on
> >their predicted outcomes.
> >Supergoal content: Events or states with "inherent" desirability, i.e.
> >desirability not contingent on predicted outcome.
>
> Shades of Kant:
> Subgoal = hypothetic imperative
> Supergoal = categorical imperative
>
> This isn't the way I would use "subgoal" and "supergoal". I would use them
> such that supergoals are "higher" level rather than inherently desirable.

That relation is described by "child goal" and "parent goal".

> I'm not convinced that there are any supergoals in the sense you intend.

If there are no supergoals, i.e., states with non-contingent desirability,
then where do the "parent goal" relations ground? Even the
good-old-fashioned Interim Goal System still revolved around the
assumption that there were supergoals *somewhere*. If you say that state
X "has no parent goal", but that its desirability is contingent "upon
state X leading to state Y", then that contradicts the definition.

-- -- -- -- --
Eliezer S. Yudkowsky http://singinst.org/
Research Fellow, Singularity Institute for Artificial Intelligence



This archive was generated by hypermail 2.1.5 : Sat Nov 02 2002 - 08:08:01 MST