From: Lee Corbin (lcorbin@rawbw.com)
Date: Mon Jun 30 2008 - 23:25:58 MDT
Martin writes
>> Tim writes
>>
>> > By the way, if the AI has any long-term goals, then it will want to
>> > preserve its own integrity in order to preserve those goals. Although
>> > "preserve its own integrity" is a good enough example for the issue at
>> > hand, it's not something you'd really need to put in there explicitly.
>
> Strong sense of identity and self-modifying are somewhat contradictions.
Yes, but we know of many devices (e.g. us) that live with contradictions.
It can be put in "society of mind" terms: a single entity may still have
competing agents (e.g. a human being barely able to make up his mind
in some particular case, a well-formed highly unified nation or group
that has internal dissent but manages to keep from displaying its dirty
laundry in public, or a simple apparatus with a built-in governer that
shudders and shakes unable to quite settle into a steady course of
action).
> (similar to drugs that change the personality)
Yes. But people often do use them. And as illogical as it is to me
(and I'm in a *really* small minority here), many people don't have
a problem with becoming quite different over time.
> That might even lead to the point that a self-aware AI does not
> want to modify itself to preserve its current state of being.
Yes, but it would then have to "care" about its existence---and
that still seems to be a hot point of contention here about how
"natural" or "necessary" it is.
> Likewise a friendly AI might be aware that the next-level
> AI it's supposed to develop might not be guaranteed friendly
> and therefore refuse to develop it at all.
Yes :-) much like we wonder if we should develop AI.
(Of course, since our six billion people are not in a single
high-integrity group that can come to a meaningful decision
about it, AI will be developed by someone, like it or not.)
> [obligatory new poster and foreign speaker disclaimer]
Welcome! I don't think I'd be able to tell that you weren't
a natural English speaker from the above writing. Thanks
for the effort.
Lee
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:01:03 MDT