From: Eliezer S. Yudkowsky (sentience@pobox.com)
Date: Wed Jan 13 1999 - 20:08:25 MST
Dick.Gray@bull.com wrote:
>
> In another thread, I and others are pointing toward a proof of the
> existence of an objective basis for ethics. Join in if you please and
> critique our arguments, don't simply dismiss them as "subjective opinions".
> (Of course, in a real sense, _all_ opinions are subjective, but that's
> obviously not the sense you meant, is it?)
I don't know of anybody who's claiming a proof for the existence of an
objective basis of ethics. How could you possibly do that without
producing the goods? All I (at least) am trying to do is establish
that, as an interim working assumption, we should assume that there's
something that can be rationally discussed - that it makes sense to
present rational arguments for pleasure being good or nuking a K-Mart
being evil.
It's entirely possible that this isn't the case. It's possible that
intelligence is totally dissociated from ultimate goals, so there isn't
an ultimate goal that's more "rational" or more "intelligent" than other
goals, and that the behavior of an SI is determined by whatever the
instinct-happy mortal programmers knowingly or unknowingly heap into the
goal system. It's equally possible that any SI goes "aha!" and accesses
an obvious, unassailable proof of a pre-existing ultimate goal totally
external from evolution and our own fevered imaginings.
But answers that might be objectively and rationally correct take
practical precedence over answers that might be arbitrarily and
indistinguishably "correct". Moreover the rules for Rational Discussion
of Questions With Real Answers External to the Participants, when
compared to the rules for Fighting Over Whose Arbitrary Answers Will Be
Imposed on Everyone Else, tend to be rather more productive of
impressive technology, not to mention creative and intelligent discussion.
--
sentience@pobox.com Eliezer S. Yudkowsky
http://pobox.com/~sentience/AI_design.temp.html
http://pobox.com/~sentience/sing_analysis.html
Disclaimer: Unless otherwise specified, I'm not telling you
everything I think I know.
This archive was generated by hypermail 2.1.5 : Fri Nov 01 2002 - 15:02:49 MST