From: William Pearson (wil.pearson@gmail.com)
Date: Wed Apr 23 2008 - 04:47:13 MDT
2008/4/23 Matt Mahoney <matmahoney@yahoo.com>:
>
> I believe the friendliness problem needs to be studied in this context (as a
> forecast) rather than as an engineering problem. When most of the computing
> power on the internet is carbon based, there is an economic incentive to
> attach specialists that serve the interests of humans. Machines compete for
> human attention in a market where information has negative value. When the
> balance of power shifts to silicon, machines will compete more for the
> attention (and computing resources) of other machines.
I think it matters more how the utility functions or equivalent
computational process(es) that determine what the systems does and
finds important are implemented (silicon/carbon) than pure compute
power.
Currently what your PC does is entirely under your control or another
humans (your can always reformat to another OS, if you dislike
windows). I think it likely that most silicon compute power will
become slaved to human desires at least to start with, due to
economics. It would be as if the silicon was an external part of the
brain. Only situations where the compute power is very separated from
humans (e.g. undersea/space exploration), is it likely to have morphic
intent per se.
Later on we might well get, "Mind Children", when we have
significantly better understanding of the brain, and people
instantiate their morphic intention* in silicon.
Will Pearson
* Morphic intent is a concept I have been playing around with, but
haven't quite polished. It is what the system desires to become. A
systems morphic intent need not be the same as the obvious intent. If
it was a young child would never grow up from wanting their favourite
book read to them. Similarly no one would become addicted to heroin.
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:01:02 MDT