Nick Bostrom wrote:
>
> It appears to me that Anders and Robin (and Max?) have a tendency to
> think that there are lots of checks and balances in the nature of
> things, and that tradeoffs and diminishing returns guarantee that a
> large variety of different strategies will always co-exist. Eliezer
> and other extreme singularians (and to some extent me) seem to think
> that the world and posthuman society might well be more brittle, and
> that system could easily end up in one of the "boring" states,
> i.e. simple states.
I would phase this another way: Either posthuman society will be maximally complex, or it will be maximally simple. If there are tradeoffs and checks and balances, they will not be the ones we know.
Human perception is the perception of differences, and we are evolved to compete in the areas where narrowly balanced forces give rise to differential rates of reproduction. This has given us a skewed and unrealistic view of the Universe as balanced between competing agents, with force and resistance always in the same order of magnitude; the unbalanced forces are not "perceived", as discrete and changeable values, at all.
Life can only evolve in the areas free from overwhelming forces and positive feedback - but balanced forces and negative feedback are really very fragile things, which is why organic life doesn't evolve in the depths of space or the heart of stars. If you want to speculate about post-Singularity processes, you have to abandon the basic human ontology. That's what makes an "extreme" Singularitarian.
The Universe is extreme, but life can evolve only in the rare balances.
--
sentience@pobox.com Eliezer S. Yudkowsky
http://pobox.com/~sentience/AI_design.temp.html
http://pobox.com/~sentience/sing_analysis.html
Disclaimer: Unless otherwise specified, I'm not telling you
everything I think I know.