Dick.Gray@bull.com wrote:
>
> In another thread, I and others are pointing toward a proof of the
> existence of an objective basis for ethics. Join in if you please and
> critique our arguments, don't simply dismiss them as "subjective opinions".
> (Of course, in a real sense, _all_ opinions are subjective, but that's
> obviously not the sense you meant, is it?)
I don't know of anybody who's claiming a proof for the existence of an objective basis of ethics. How could you possibly do that without producing the goods? All I (at least) am trying to do is establish that, as an interim working assumption, we should assume that there's something that can be rationally discussed - that it makes sense to present rational arguments for pleasure being good or nuking a K-Mart being evil.
It's entirely possible that this isn't the case. It's possible that intelligence is totally dissociated from ultimate goals, so there isn't an ultimate goal that's more "rational" or more "intelligent" than other goals, and that the behavior of an SI is determined by whatever the instinct-happy mortal programmers knowingly or unknowingly heap into the goal system. It's equally possible that any SI goes "aha!" and accesses an obvious, unassailable proof of a pre-existing ultimate goal totally external from evolution and our own fevered imaginings.
But answers that might be objectively and rationally correct take practical precedence over answers that might be arbitrarily and indistinguishably "correct". Moreover the rules for Rational Discussion of Questions With Real Answers External to the Participants, when compared to the rules for Fighting Over Whose Arbitrary Answers Will Be Imposed on Everyone Else, tend to be rather more productive of impressive technology, not to mention creative and intelligent discussion.
--
sentience@pobox.com Eliezer S. Yudkowsky
http://pobox.com/~sentience/AI_design.temp.html
http://pobox.com/~sentience/sing_analysis.html
Disclaimer: Unless otherwise specified, I'm not telling you
everything I think I know.