From: Eliezer S. Yudkowsky (sentience@pobox.com)
Date: Fri May 31 2002 - 22:05:42 MDT
Wei Dai wrote:
>
> On Thu, May 30, 2002 at 02:28:07PM -0400, Eliezer S. Yudkowsky wrote:
> > But there are many different factors on which any of us can be prejudged. I
> > can be prejudged as having more than two Jewish grandparents, as a white
> > male, as a Georgian (male), as a twenty-something (male), or many other
> > stereotypes which would be utterly misleading to anyone trying to sketch out
> > a picture of Eliezer Yudkowsky. There are not "A" players and "B" players.
>
> I thought you were saying that even rational Bayesians
Rational Bayesian *agents*. They aren't just modelling the world, they're
playing games and making contracts with other agents in a multiplayer
non-zero-sum reputation-based system.
> have game
> theoretical reasons to not prejudge people. (That would be really cool if
> it was true, but I don't see how it could be true.) If you're just saying
> that we shouldn't prejudge people based on misleading stereotypes,
If there were an infallible way or even *any* way to tell when stereotypes
(group-based statistical associations) were "misleading", they wouldn't be
stereotypes.
> then of
> course I agree with you, but that doesn't seem to have anything to do with
> the Prisoner's Dilemma.
>
> So could you please clarify, are there or are there not game theoretical
> reasons for Bayesian reasoners to not prejudge? If the answer is yes,
> please provide an example to show what the reasons are.
There are ethical reasons (in the CFAI sense) and game-theoretical reasons
(in the standard sense) for rational humans to not prejudge. I'm not sure
if Bayesian reasoners would run into the problems to which "not prejudging"
is a rational solution, but if you're a human living in a society of humans
then there are various forces you need to compensate for.
One obvious ethical reason not to prejudge is that, for humans, this is the
mental equivalent of sticking your head in your microwave. Gonna try and
think rationally about group membership with ten million years of warring
tribes inscribed onto your genes? Seems unlikely. Best to stick to being a
member of the "sentience" tribe. But let's leave that aside for a moment.
We're humans. We've been playing reputation-based games for a long, long
time, and we have strong emotions related to reputation. Being judged for
something you didn't do is profoundly frustrating. Being judged based on a
nonvolitional (unchosen) surface characteristic that happens to be shared by
someone *else* who did something bad is, obviously, a case of being judged
based on something you didn't do and have no control over. So one obvious
sense in which refraining from prejudgement can be a "trade" is if two
people wish to exchange avoidance of frustration. Refraining from this kind
of prejudgement can also be a public good. Someone paying the penalty for
someone else's defection may feel that, if they have to pay the penalty
anyway, they may as well defect themselves. In a reputation-based system,
the integrity of the reputation system is a public good.
I'm not quite sure whether the same kinds of problems would arise among pure
Bayesian systems - but among humans, who are prone to make "sticky"
group-based prejudgements that last even in the face of qualitatively and
quantitatively stronger contrary evidence, and who have strong emotions
about both how to judge and what it means to be judged, preassignment of
game-theoretical reputation is far more dangerous than simple Bayesian
adjustment of evidence.
-- -- -- -- --
Eliezer S. Yudkowsky http://singinst.org/
Research Fellow, Singularity Institute for Artificial Intelligence
This archive was generated by hypermail 2.1.5 : Sat Nov 02 2002 - 09:14:32 MST