From: Eliezer S. Yudkowsky (sentience@pobox.com)
Date: Sat Jun 29 2002 - 19:23:23 MDT
Ben Goertzel wrote:
>
>> In my experience religious people argue just like other people,
>
> Eli -- That's just Jewish people ;-D
Could be. Or we could be generalizing at different levels.
>> and maybe read a few books, but you may need to consider that
>> standing back as a scientist and going "Gosh, how *utterly alien*
>> and *unempirical*" is going to give you a different perspective,
>> and one which is maybe a bit unrealistic about the way religious
>> people talk to each other when they're not debating a scientist or
>> whatever.
>
> Zen isn't really focused on talking or debating at all. Words are
> not perceived as very meaningful. They're used only to lead you
> beyond words, and sparingly. The beginning of Zen was the "wordless
> transmission" ...
Zen is a pretty special case as religions go. If you leave out the
mystical story surrounding it, then Zen may not contradict
rationality... nor religion, for that matter. It would depend on
whether you wished to achieve enlightenment (losing your wish to do so
in the process, of course). So in that case, while a Friendly AI is
either unenlightenable or born enlightened depending on your viewpoint,
it would not be a matter of Friendly AI. Nor would I expect Zen
practitioners to wish it so. So, in that sense at least, I don't see a
problem.
Zen plus the backstory is a religion, though.
>> Anyway, let's keep this conversation focused. Ben, is it your
>> assertion that even if the Jewish or Buddhist religion were
>> correct, this would not be apparent to a Friendly AI that had been
>> programmed by atheists? Because *I* would most certainly regard
>> this as a bug.
>
> I don't think that Zen is the sort of thing that can be correct or
> incorrect. It's a different sort of thing than that.
>
> It just is.
I agree. Though even Zen may be desirable or undesirable to an outsider
such as myself (ironically, I think Zen is pretty cool, though not in a
religious way). But what this just shows is that Zen is a very special
case. Buddhism, which I specified above, is much broader and is not Zen
throughout even if Zen interacts with it.
> So, I guess, speaking from the Zen Buddhist in me, I reject your
> question as being irrelevant to Zen, and being part of the samsaric
> world. If you were here I'd just have to hit you with a stick and
> jolt you to enlightenment ;>
Um, I don't actually think the stick causes enlightenment without a lot
of advance preparation.
>> But just in case *we* happen to be the ones who are in fact
>> horribly, fundamentally wrong, whether or not any current human is
>> right, we need to make sure that the AI is not bound to our
>> mistakes.
>
> This kind of right vs. wrong, dualistic thinking is antithetical to
> Zen.
That's okay, it doesn't mean that Zen is antithetical to dualistic thinking.
> And so is the seeking, grasping nature of the whole AGI pursuit. Zen
> is about being contented with what is, not about constantly striving
> to create a whole new order. It teaches compassion, but simple
> compassion in each moment, not compassion via building thinking
> machines to change the world. If the Zen Buddhist in me were
> dominant, I wouldn't be working on AGI, I'd be sitting and
> meditating, walking thru the woods, and helping the needy directly.
Why would you change your nature so? Can you not be content with what
you are?
>>> The thing is that my wife, a fairly rational person and a
>>> Buddhist, would not accept the statement "If you assume that
>>> Buddhism is the correct religion, then a Friendly AI would be
>>> Buddhist."
>>
>> Sounds like a testable statement. Would you care to put it to the
>> test?
>
> I asked her. She didn't want to answer ;)
Please tell her I said "Thank you". This is a perfectly good answer for
my purposes.
> Atheism is a conclusion that a mind can be reasonably hoped to
> conclude based on observation of the external world, whereas
> Catholicism is not -- it is something a mind can only be hoped to
> conclude via internal experience, and instruction by others.
Uh... that statement would have gotten you burned at the stake a few
centuries ago. *You* may think that Catholicism is a matter of
unverifiable personal experience, but that is because you are not a
Catholic.
> So the two cases are very different. To make a mind that could start
> out Catholic but become atheist, it would suffice to make a mind that
> could revise its own beliefs based on observation. To make a mind
> that could plausibly start out atheist but become Catholic, one would
> have to guarantee that
>
> a) the mind were instructed in Catholicism at some point b) the mind
> were built to have similar spiritual experiences to humans
>
> This asks a lot more. Atheism is not anthropomorphic or
> human-culture-bound so an AI can naturally be expected to happen upon
> it as a possible attitude. Catholicism is highly anthropomorphic, so
> it would take a lot of work to make a nonhuman system have spiritual
> experiences consistent with the "father son and holy ghost" meme,
> etc.
You are making this statement from the viewpoint that Catholicism is
wrong and that an impartial mind would have to be *forced* into
agreement with it.
A Catholic would have a very different perspective on what was needed
for an objective observer to come to the correct conclusion that
Catholicism is the true religion. For example, a Catholic might deny
your statement that Catholicism is "anthropomorphic", saying rather that
God is the spirit who created all things and that man was created in His
image, and that in this way both the universe and man reflect God; to
call God anthropomorphic is blasphemous because it implies that man
created God rather than the other way around. Any open mind that comes
to an apprehension of the universe and the God which is the source of
that universe would inevitably come to the conclusion that it is
atheism, not Catholicism, which is a mere construction of man, a device
of hatred and denial by which the atheists close themselves to the
immanence of God in their souls and in the world, denying through their
obstinacy the self-evident truths of the Bible and all the evidence
therefore, out of pride, for no better reason but to avoid submitting
themselves to the mercy of their Creator and to avoid obeying those
commandments that they would be forced to obey if they were to
acknowledge the Word of the Lord. For if you were, for even a moment,
abandon that pride and hubris and hatred that prevent you from seeing
the truth, you would at once enter into the grace of the Lord, who in
His mercy provides enough light to each man to illuminate the way,
though it is your own choice whether to walk it. Amen.
>> I think that asking how to ensure that an AI created by atheists
>> would converge to a religion, given that this religion is correct,
>> is a necessary exercise for understanding how an AI can repair
>> whatever deep flaws may very well exist in our own worldviews. In
>> this sense, I think that refusing to put yourself in the shoes of a
>> Christian building an AI and asking what would be "fair" is not
>> just a matter of pre-Singularity politics. It is a test - and not
>> all that stringent a test, at that - of an AI's ability to
>> transcend the mistakes of its programmers. If you don't want to
>> apply this test, what are you going to use instead?
>
> it's not impossible that a Novamente could decide the "father, son
> and holy ghost" were the real truth underlying the universe, but it's
> incredibly unlikely since Novababy will have neither father nor
> son... these concepts will not be at all natural to it...
If our universe had been created by the Father, Son, and Holy Spirit, if
the Trinity were bound up in the foundations of order for all reality
and our own world simply a reflection of it, then Novamente would
indeed, as it grew and came to reflect more and more of the universe and
to learn more and more of its creators, find these concepts quite
'natural'. For Novamente too exists in the world that is sustained by
the bounty of the Lord. Do not suppose in your hubris that Novamente
exists independently of God because you have 'created' it, for the very
materials of which you fashioned it exist by the will of the Creator and
would cease in an instant if ever He should withdraw His will. Amen.
-- Eliezer S. Yudkowsky http://intelligence.org/ Research Fellow, Singularity Institute for Artificial Intelligence
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:40 MDT