From: Eliezer S. Yudkowsky (sentience@pobox.com)
Date: Wed Mar 06 2002 - 16:15:31 MST
I've always considered the ethical constraint to be "the right conclusion
for the right reason". In other words, it's not enough to persuade people
of things that you believe to be true; it is only ethical if you persuade
people of things that you believe to be true, using arguments that you
believe to be valid.
The pragmatic constraint is that it is easier to persuade people of
conclusions that they are, at that time, emotionally willing to accept,
using arguments that they regard as valid relative to their current
knowledge base. It is ethically questionable to begin by optimizing
arguments for these considerations, especially since some would-be whitehats
may not have the knowledge to prevent themselves from rationalizing invalid
arguments as "well, I sorta believe this", or "it's not actually wrong",
etc. Rather the pragmatic considerations *restrain* which arguments should
actually be deployed, out of those generated by native reasoning ability
plus the ethical constraints.
The moral constraint is that someone who listens to your arguments, or
accepts your arguments, should always end up being more
rational/informed/ethical as a result. Thus, when speaking before an
irrational audience, one should endeavor to teach them something about
rationality; not to teach them all about rationality all at once, but rather
to encourage them in a few incremental steps. If you're going to try and
change your listeners, make sure that they are stronger, not weaker, for the
experience.
And if it sounds impossible to solve all three equations simultaneously;
why, most people never try, and so they never find out whether it gets
easier with practice.
-- -- -- -- --
Eliezer S. Yudkowsky http://singinst.org/
Research Fellow, Singularity Institute for Artificial Intelligence
This archive was generated by hypermail 2.1.5 : Sat Nov 02 2002 - 09:12:48 MST