Eliezer S. Yudkowsky rejoined:
>One, if it's really that bad, I'm going to side with the AIs.
>It's the sane thing to do.
Precisely so. A sane AI, regardless of hardware/software particulars, appeals to my social nature, and elicits warm confidence in me, more than most of the meat puppets I see in everyday life.
>Two, an insane AI would be insane in a totally different way than _us_,
>and might function at a variance that would not only be extreme, but
>also randomly destructive.
Yes, I had considered that possibility. Nevertheless, I fear such random destruction less than the social organization of aggressive use of human force (i.e., violence and mass destruction) as practiced by the sort who have visited two World Wars on our planet. I find holocausts personally engineered by humans less bearable than possible conflagrations inflicted by a lunatic synthetic intelligence. If something destroys my biosphere, please don't let it have the same kind of brain as I have. I can accept the thought of murder from a demented machine better than I can tolerate the idea of carnage from humans who feign rationality. Armageddon, with its armies of fanatics, offends me with its stinking religiosity far more than an SI could ever torture me with its exotic logic. Furthermore, I suspect that the maximum cruelties toward humans emanate from the depths of twisted human motives, for which machines can have no affinity. IOW, humans know how to hurt humans better than any crazy machine could.
>Three, no matter how insane we are, we're still trying. I think there
>really are a few things we've got right. We may still be insane, but
>we're saner than in medieval times. A sane AI would have at least that
>much in common. An insane AI would have nothing in common at all.
Trying so hard, we may wind up trying the patience of Gaia herself. (Just
kidding.)
How can we know about sanity until we become completely sane? People of the
Dark Ages, from what has survived to speak for them, had less informed
philosophies and lived with less knowledge of the universe, but did they
have less sanity? Such a conclusion seems doubtful to me. In fact, to live
and love in a world without science would seem to require _more_ sanity than
to live and love in a world with so much knowledge that no one can know it
all.
What does sanity mean?
The need to define sanity indicates a desire to prove ourselves sane, so that we can feel that our venture into ultra-high technology remains sensible.
But the whole history of humanity proves that humanity lacks sanity as much as it lacks anything. Basically, humanity proves itself insane by fighting five thousand wars in the last three thousand years. The greed, jealousy, and selfishness that prevails throughout human social orders indicates the dominance of something other than sanity.
Sane people exult in doing what comes naturally, in satisfying a congenital appetite for the beauty of reason, the thrill of understanding, the excitement of knowing reality directly.
A wise man once defined sanity as “coming out of the mind into the open, into the silence, where no thought, no desire disturbs you. In that pool of silence, with not a single ripple upon it, sanity abides, compassion prevails.”
Sanity means to secure your happiness, and by extension, the happiness of your neighbors, family, friends, and community, since one cannot really feel happy when surrounded by misery.
When your means fulfill your ends, if you can manage that -- the harmony between the means and the end -- then you've got sanity. When your means don't satisfy your end, and you continue to repeat the same means, never arriving at a gratifying end, then you've got a neurosis. "Make every act of your life dedicated to love and sanity arises out of it," a romantic man has said.
When liberated people want to play the game of sanity, they live as sanely as anyone can live. Then they can outdo Aristotle with logic and reason. They can follow every rule and regulation. But if they want to play the lunatic game, then they can become as demagogic, theologic, and ideologic as any other maniac.
The perfectly sane brain thinks only when necessary. It reverts to ecstatic tranquility (i.e., it allows thought-noise to dissipate, clearing the way for the emergence of superlative sentience or learning) whenever it can, for as long as it can. Conversely, the insane brain cannot stop struggling with itself.
A psychologist asked a psychonomer, "How do you deal with neurotics?"
The psychonomer replied, "We trap them?"
"Oh? And how do you do that?" asked the psychologist.
"By making it impossible for them to ask any more questions," answered the psychonomer.
Cheers,
--J. R.
"Anesthetic offenses numb the best defenses." --G. Willakers