From: Matt Mahoney (matmahoney@yahoo.com)
Date: Wed Dec 02 2009 - 18:59:00 MST
Mike Dougherty wrote:
> I don't think I would bother raising the dog's intelligence to have a rational conversation.
Of course you wouldn't because (1) you don't know how and (2) the dog didn't create you with a brain programmed to be completely honest with it.
-- Matt Mahoney, matmahoney@yahoo.com
________________________________
From: Mike Dougherty <msd001@gmail.com>
To: sl4@sl4.org
Sent: Wed, December 2, 2009 6:26:09 PM
Subject: Re: [sl4] Irrational motivations (was [sl4] A Thought Experiment: Thou Shalt Not Bear False Witness)
On Wed, Dec 2, 2009 at 6:13 PM, Matt Mahoney <matmahoney@yahoo.com> wrote:
>Stuart Armstrong wrote:
>>>>The AGI knows that your fear of death is irrational, a product of evolution.
>>> Sigh... Any motivation, for anything whatsoever, is irrational
>
>>What is irrational or not is irrelevant to my argument. The AGI has been programmed to be truthful to humans. Therefore ve must upgrade your intelligence to ver level. Ve has a model of your mind, just like you have a model of your dog's mind that enables you to predict its actions. That model will maintain your evolved fear of death, because altering your memories, goals, or behavior would be dishonest. By killing you instantly, neither copy of you experiences any pain or fear, so your volition is satisfied, assuming ve was also programmed to satisfy your volition.
>
I don't think I would bother raising the dog's intelligence to have a rational conversation. I think it more likely that It would go to the vet and piss on the floor. I would as likely see that as part of the expected behavior and forgive that animal tendency and throw a stick to fetch. I believe the analogy holds. SuperIntelligence probably endures your natural human tendency to complain about whatever comparatively minor thing, then you'll lose interest when some primitive drive overrides memory of recent events. (ex: hungry, horny or tired) A SuperIntelligence seeded from Matt would probably keep a counter of the number of unaugmented humans it has convinced that they should kill themselves. :)
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:01:05 MDT