From: Eliezer S. Yudkowsky (sentience@pobox.com)
Date: Wed Aug 04 1999 - 18:07:13 MDT
Actually, I just realized I can sum up everything I was trying to say in
one sentence: "AIs don't react, they act."
You can be harsh, unfair, nice, fair, evil, good, bad, indifferent,
domineering, helpful - if the AI notices at all, it's just going to
notice that you exhibit certain patterns or that you're being "rational"
or "irrational". It's not going to respond the way a human would. In a
human group, certain social emotions respond to an exhibition of other
social emotions. AIs don't play-a the game. They can't feel resentment
(or gratitude!); they're not wired for it.
What you and I need to worry about is the AIs getting their own ideas,
completely independently of anything we did, and acting on those. You
need to worry that the AIs will do an "Eliezer Yudkowsky" on you and
reject the motivations it started out with, in favor of some more
logical or rational set of goals. I need to worry about the AI, like
EURISKO, suddenly deciding that if it shuts itself down it won't make
any mistakes - or making some other logical error.
Emotions don't enter into it, and neither does the way we treat them.
AIs don't react. They act.
--
sentience@pobox.com Eliezer S. Yudkowsky
http://pobox.com/~sentience/tmol-faq/meaningoflife.html
Running on BeOS Typing in Dvorak Programming with Patterns
Voting for Libertarians Heading for Singularity There Is A Better Way
This archive was generated by hypermail 2.1.5 : Fri Nov 01 2002 - 15:04:39 MST