From: Anders Sandberg (asa@nada.kth.se)
Date: Tue May 21 2002 - 10:13:23 MDT
On Tue, May 21, 2002 at 10:10:59AM -0400, Eliezer S. Yudkowsky wrote:
> I don't think any human programmer could possibly defend a network against a
> seed AI. That kind of programming ability is simply out of our reach.
> Humans do not write *flawless code*.
It is also unlikely AIs will write flawless code, unless they are very
smart AIs and it is very simple code. Which suggests that one defence
against roaming seeds is to have semi-smart systems protect the network:
it seems to me that the issue of protecting a computer from takeover is
a far more well-defined and focused problem than being a seed AI, so AI
technologies used on this problem would likely give a larger security
payoff. They don't have to be perfect, just good and distributed enough
so that the probability of a small blight sneaking through becomes
negligible. That takes care of a lot of risk, and by not trying to make
the defenses supersmart there is no risk that they become general
blights too.
I'm also reminded of the paper by Traniello, Rosengaus and Savoie in
last issue of PNAS (vol 88:10 6838-6842) about disease resistance in
termites. It turns out that the termites are more disease resistant when
they are in groups than when alone. While this may be due to them being
weakened by isolation, some factors cast doubts on this (they do fine in
isolation on many other immune-issues). So maybe the termites are
running a kind of P2P immune system, where they help each other become
more resistant to pathogens than they would be alone. The same
mechanisms might be interesting to explore in a computer setting: can we
create intrusion countermeasures and antiviral software that is
leveraged by P2P?
-- ----------------------------------------------------------------------- Anders Sandberg Towards Ascension! asa@nada.kth.se http://www.nada.kth.se/~asa/ GCS/M/S/O d++ -p+ c++++ !l u+ e++ m++ s+/+ n--- h+/* f+ g+ w++ t+ r+ !y
This archive was generated by hypermail 2.1.5 : Sat Nov 02 2002 - 09:14:14 MST