From: J. Eric Mortensen (eric@episteme.no)
Date: Thu Feb 25 1999 - 14:54:51 MST
Here's an idea:
The short version:
Assuming that SI's are similar to humans in kind of sentience,
they will possibly develop (among themselves) norms not too different from our norms
(do not kill, do not steal etc.). They will possibly also develop an empathy for other
intelligent beings, given enough time of interaction with us. The way to instill the
best moral valus (both seen from their own perspective and ours) in SIs, might then
be to have them "genetically" evolve cooperative (tit-for-tat) strategies which will incline
them to be nice to us (and each other). Such evolution might constitute the pre-programming
of each SI. An SI will, when born, be the product of a simulated evolution, which will in theory
give him or her the guiding intuition/instinct that he/she needs to live a good life.
Whether or not he fully comprehends his own behavior is not important, in the same way it
hasn't been very important for civilization to grow that we humans know why murder is bad. (We
just know that if we kill our neighbor it's more than likely that his friends and family will kill us.)
A longer version:
Resarch in cooperation theory (Axelrod etc.) indicates that being nice to others
(following a variant of the tit-for-tat-strategy) is a very likely strategy to emerge
in a population of interacting beings that each are in a prisoner's dilemma
toward the others. Human norms and culture (and to an extent its implication,
human morality) can be thought of as the result of such developments.
True, humans are not perfect and they can
be brutal towards one another, but I still know of no society where murdering your friendly
neighbor is not in some way punished. Ditto for theft, kidnapping etc.
If SI's are sentient in a way not to dissimilar to the way humans are sentient, I think it's
reasonable to think that superintelligent beings will develop much the same norms as
humans wrt how to behave toward each others. After all, if they are self-reflective, rational,
optimizing, moderately egoist, empathizing individuals, they will see each other much the
same way we humans see each others, as members of a fellow race, with moderately
predictable behavior with whom they can have valuable relationships, but also with whom they
can have costly conflicts if not checked carefully.
Furthermore, it seems that the more we humans remove ourselves from our brutish origins the
more we seem to develop empathy towards beings of other (animal) races.
Witness how the question of animal rights is becoming more and more important for every
decade. As long as SI's are not overwhelmingly powerful or different (so as to make
communication difficult) compared to humans, they should also see us as valuable social
partners. Among all the conceivable strategies that SI's and humans might play against each
others, the fact that we are fairly similar (by assumption) and the fact that we will tend to be
social to members of our own group, should lead to both races discovering mutually optimal
strategies, i.e., cooperation.
I'm probably too anthropomorphic in my thinking, but the basic idea is that if human behavior is
determined more by evolution of adapted strategies than rational optimization, and if SI's are preprogrammed to behave in much the same way, the outcome should be that SIs will develop
the wanted moral values (be nice to others etc.).
Any comments?
Regards,
Eric
This archive was generated by hypermail 2.1.5 : Fri Nov 01 2002 - 15:03:08 MST