From: Billy Brown (bbrown@conemsco.com)
Date: Wed Feb 24 1999 - 14:59:16 MST
Nick Bostrom wrote:
> Does this mean you think that no animals are sentient? Sounds
> implausible to me.
No, it just means I'm trying to avoid starting an argument about animal
sentience by phrasing the issue narrowly.
> No, at least that is not what I am proposing. Let it be able to think
> about morality. Let it also be able to change its fundamental values.
> If I am right then that won't matter, because it will not *want* to
> change them. (I'm almost tempted to defin a "fundamental value" as: a
> preference that you would not want to change.) What I am suggeting is
> that any SI we build has repect for human rights as a fundamental
> value. As long as we make sure it has that value, then we need have
> nothing to fear. It will go about its business and perhaps transform
> itself into a power beyond all human understanding, but it would not
> harm us humans, because it would not want to harm us. Maybe speaking
> of an "inviolable moral code as a core element of its programming"
> conjures up the wrong conotations -- as if it were some form of
> coercion going on. I see it simply as selecting one type of value
> (human-friendly) rather than another (indifferent or hostile)..
Ah. I see. I think its the word "inviolable" that does it.
If you simply want to make sure they think morality is important, and that
they start out with a solid understanding of human ideas about morality,
then I think you've got the right idea. If you want to devote special
effort to ensuring that a few core ideas, like "don't kill people without a
really, really pressing reason" are given a high level of priority, that is
only reasonable. I think there are real dangers in getting overambitious
about this sort of thing, but that's a different issue.
Perhaps the line could be re-worded to avoid this confusion? Something like
"In the first case, we could make sure that they possess a thorough
understanding of, and respect for, existing human moral codes that emphasize
respect and toleration of others." would seem to convey your intended
meaning without the possibility of confusion.
Billy Brown, MCSE+I
bbrown@conemsco.com
This archive was generated by hypermail 2.1.5 : Fri Nov 01 2002 - 15:03:08 MST