From: Peter de Blanc (peter@spaceandgames.com)
Date: Tue Oct 30 2007 - 22:08:30 MDT
On Wed, 2007-10-31 at 10:20 +1100, Stathis Papaioannou wrote:
> We take risks all the time. Crossing the street to buy an ice-cream
> puts you at increased risk of being hit by a car and of cardiovascular
> disease. If a million people do this, let's say at least one person
> will almost certainly die. Is it reasonable to sacrifice one human
> life for something as trivial as an ice-cream? To an AI that can see
> these probabilities as clearly as you can see a car about to hit a
> pram, the answer may be "no". But taking such an attitude towards all
> the risks that humans face many times a day may make life not worth
> living.
I think you're imagining an AI that implements some, but not all, of
human morality. This is the real problem. If an increase in optimization
power is hurting you, then you're optimizing for the wrong thing.
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:58 MDT