[p2p-research] Applying utility functions to humans considered harmful

Sun Feb 7 23:04:10 CET 2010

I see where the founder of Second Life is starting a new AGI firm where his
goal is the singularity.  Interesting times in AI (after a long dark
period--what is regularly now called the AI Winter.)

Ryan

On Wed, Feb 3, 2010 at 9:20 PM, Michel Bauwens <michelsub2004 at gmail.com>wrote:

> the pdf won't load here, but that seems to be the Van Gelder I was
> referring to in a discussion with jandrews, as saying that equating brains
> and humans as machines belonged to the infantile phase of AI,
>
> Michel
>
> On Thu, Feb 4, 2010 at 8:21 AM, Ryan <rlanham1963 at gmail.com> wrote:
>
>> For the utilitarians out there...
>>
>>
>>
>> Sent to you by Ryan via Google Reader:
>>
>>
>> Applying utility functions to humans considered harmful<http://lesswrong.com/lw/1qk/applying_utility_functions_to_humans_considered/>
>>  via lesswrong: What's new <http://lesswrong.com/> on 2/3/10
>>
>> Submitted by Kaj_Sotala <http://lesswrong.com/user/Kaj_Sotala> 19
>> comments<http://lesswrong.com/lw/1qk/applying_utility_functions_to_humans_considered/#comments>
>>
>> There's a lot of discussion on this site that seems to be assuming
>> (implicitly or explicitly) that it's meaningful to talk about the utility
>> functions of individual humans. I would like to question this assumption.
>>
>> To clarify: I don't question that you couldn't, *in principle*, model* *a
>> human's preferences by building this insanely complex utility function. But
>> there's an infinite amount of methods by which you could model a human's
>> preferences. The question is which model is the most useful, and which
>> models have the least underlying assumptions that will lead your intuitions
>> astray.
>>
>> Utility functions are a good model to use if we're talking about designing
>> an AI. We want an AI to be predictable, to have stable preferences, and do
>> what we want. It is also a good tool for building agents that are immune to
>> Dutch book tricks. Utility functions are a bad model for beings that do not
>> resemble these criteria.
>>
>> To quote Van Gelder (1995)<http://people.bu.edu/pbokulic/class/vanGelder-reading.pdf>
>> :
>>
>> Much of the work within the classical framework is mathematically elegant
>> and provides a useful description of optimal reasoning strategies. As an
>> account of the actual decisions people reach, however, classical utility
>> theory is seriously flawed; human subjects typically deviate from its
>> recommendations in a variety of ways. As a result, many theories
>> incorporating variations on the classical core have been developed,
>> typically relaxing certain of its standard assumptions, with varying degrees
>> of success in matching actual human choice behavior.
>>
>> Nevertheless, virtually all such theories remain subject to some further
>> drawbacks:
>>
>> (1) They do not incorporate any account of the underlying motivations that
>> give rise to the utility that an object or outcome holds at a given time.
>> (2) They conceive of the utilities themselves as static values, and can
>> offer no good account of how and why they might change over time, and why
>> preferences are often inconsistent and inconstant.
>> (3) They offer no serious account of the deliberation process, with its
>> attendant vacillations, inconsistencies, and distress; and they have nothing
>> to say about the relationships that have been uncovered between time spent
>> deliberating and the choices eventually made.
>>
>> Curiously, these drawbacks appear to have a common theme; they all
>> concern, one way or another, *temporal* aspects of decision making. It is
>> worth asking whether they arise because of some deep structural feature
>> inherent in the whole framework which conceptualizes decision-making
>> behavior in terms of calculating expected utilities.
>>
>> One model that attempts to capture actual human decision making better is
>> called *decision field theory*. (I'm no expert on this theory, having
>> encountered it two days ago, so I can't vouch for how good it actually is.
>> Still, even if it's flawed, it's useful for getting us to think about human
>> preferences in what seems to be a more realistic way.) Here's a brief
>> summary of how it's constructed from traditional utility theory, based on Busemeyer
>> & Townsend (1993) <http://mypage.iu.edu/%7Ejbusemey/psy_rev_1993.pdf>.
>> See the article for the mathematical details, closer justifications and
>> different failures of classical rationality which the different stages
>> explain.
>>
>> *Stage 1: Deterministic Subjective Expected Utility (SEU) theory.*Basically classical utility theory. Suppose you can choose between two
>> different alternatives, A and B. If you choose A, there is a payoff of 200
>> utilons with probability S1, and a payoff of -200 utilons with probability
>> S2. If you choose B, the payoffs are -500 utilons with probability S1 and
>> +500 utilons with probability S2. You'll choose A if the expected utility of
>> A, S1 * 200 + S2 * -200 is higher than the expected utility of B, S1 * -500
>> + S2 * 500, and B otherwise.
>>
>> *Stage 2: Random SEU theory. *In stage 1, we assumed that the
>> probabilities S1 and S2 stay constant across many trials. Now, we assume
>> that sometimes the decision maker might focus on S1, producing a preference
>> for action A. On other trials, the decision maker might focus on S2,
>> producing a preference for action B. According to random SEU theory, the
>> attention weight for variable S*i* is a continous random variable, which
>> can change from trial to trial because of attentional fluctuations. Thus,
>> the SEU for each action is also a random variable, called the *valence*of an action. Deterministic SEU is a special case of random SEU, one where
>> the trial-by-trial fluctuation of valence is zero.
>>
>> *Stage 3: Sequential SEU theory.* In stage 2, we assumed that one's
>> decision was based on just one sample of a valence difference on any trial.
>> Now, we allow a sequence of one or more samples to be accumulated during the
>> deliberation period of a trial. The attention of the decision maker shifts
>> between different anticipated payoffs, accumulating weight to the different
>> actions. Once the weight of one of the actions reaches some critical
>> threshold, that action is chosen. Random SEU theory is a special case of
>> sequential SEU theory, where the amount of trials is one.
>>
>> Consider a scenario where you're trying to make a very difficult, but very
>> important decisions. In that case, your inhibitory threshold for any of the
>> actions is very high, so you spend a lot of time considering the different
>> consequences of the decision before finally arriving to the (hopefully)
>> correct decision. For less important decisions, your inhibitory threshold is
>> much lower, so you pick one of the choices without giving it too much
>> thought.
>>
>> *Stage 4: Random Walk SEU theory. *In stage 3, we assumed that we begin
>> to consider each decision from a neutral point, without any of the actions
>> being the preferred one. Now, we allow prior knowledge or experiences to
>> bias the initial state. The decision maker may recall previous preference
>> states, that are influenced in the direction of the mean difference.
>> Sequential SEU theory is a special case of random walk theory, where the
>> initial bias is zero.
>>
>> Under this model, decisions favoring the status quo tend to be chosen more
>> frequently under a short time limit (low threshold), but a superior decision
>> is more likely to be chosen as the threshold grows. Also, if previous
>> outcomes have already biased decision A very strongly over B, then the mean
>> time to choose A will be short while the mean time to choose B will be long.
>>
>> *Stage 5: Linear System SEU theory. *In stage 4, we assumed that previous
>> experiences all contribute equally. Now, we allow the impact of a valence
>> difference to vary depending on whether it occurred early or late (a primacy
>> or recency effect <http://en.wikipedia.org/wiki/Serial_position_effect>).
>> Each previous experience is given a weight given by a growth-decay rate
>> parameter. Random walk SEU theory is a special case of linear system SEU
>> theory, where the growth-decay rate is set to zero.
>>
>> *Stage 6: Approach-Avoidance Theory. *In stage 5, we assumed that, for
>> example, the average amount of attention given to the payoff (+500) only
>> depended on event S2. Now, we allow the average weight to be affected by a
>> another variable, called the goal gradient. The basic idea is that the
>> attractiveness of a reward or the aversiveness of a punishment is a
>> decreasing function of distance from the point of commitment to an action.
>> If there is little or no possibility of taking an action, its consequences
>> are ignored; as the possibility of taking an action increases, the attention
>> to its consequences increases as well. Linear system theory is a special
>> case of approach-avoidance theory, where the goal gradient parameter is
>> zero.
>>
>> There are two different goal gradients, one for gains and rewards and one
>> for losses or punishments. Empirical research suggests that the gradient for
>> rewards tends to be flatter than that for punishments. One of the original
>> features of approach-avoidance theory was the distinction between rewards
>> versus punishments, closely corresponding to the distinction of positively
>> versus negatively framed outcomes made by more recent decision theorists.
>>
>> *Stage 7: Decision Field Theory. *In stage 6, we assumed that the time
>> taken to process each sampling is the same. Now, we allow this to change by
>> introducing into the theory a time unit *h*, representing the amount of
>> time it takes to retrieve and process one pair of anticipated consequences
>> before shifting attention to another pair of consequences. If *h* is
>> allowed to approach zero in the limit, the preference state evolves in an
>> approximately continous manner over time. Approach-avoidance is a spe... you
>> get the picture.
>>
>>
>> ------------------------------
>>
>>
>>
>> Now, you could argue that all of the steps above are just artifacts of
>> being a bounded agent without enough computational resources to calculate
>> all the utilities precisely. And you'd be right. And maybe it's meaningful
>> to talk about the "utility function of humanity" as the outcome that occurs
>> when a CEV-like entity calculated what we'd decide if we could collapse
>> Decision Field Theory back into Deterministic SEU Theory. Or maybe you just
>> say that all of this is low-level mechanical stuff that gets included in the
>> "probability of outcome" computation of classical decision theory. But which
>> approach do you think gives us more useful conceptual tools in talking about
>> modern-day humans?
>>
>> You'll also note that even DFT (or at least the version of it summarized
>> in a 1993 article) assumes that the payoffs themselves do not change over
>> time. Attentional considerations might lead us to attach a low value to some
>> outcome, but if we were to actually end up in that outcome, we'd always
>> value it the same amount. This we know to be untrue. There's probably some
>> even better way of looking at human decision making, one which I suspect
>> might be very different from classical decision theory.
>>
>> So be extra careful when you try to apply the concept of a utility
>> function to human beings.
>>
>>
>>
>> Things you can do from here:
>>
>>    - Subscribe to lesswrong: What's new<http://www.google.com/reader/view/feed%2Fhttp%3A%2F%2Flesswrong.com%2F.rss?source=email>using
>>    *Google Reader*
>>    - Get started using Google Reader<http://www.google.com/reader/?source=email>to easily keep up with
>>    *all your favorite sites*
>>
>>
>>
>>
>> _______________________________________________
>> p2presearch mailing list
>> p2presearch at listcultures.org
>> http://listcultures.org/mailman/listinfo/p2presearch_listcultures.org
>>
>>
>
>
> --
> Work: http://en.wikipedia.org/wiki/Dhurakij_Pundit_University - Think
> thank: http://www.asianforesightinstitute.org/index.php/eng/The-AFI
>
> P2P Foundation: http://p2pfoundation.net  - http://blog.p2pfoundation.net
>
> Connect: http://p2pfoundation.ning.com; Discuss:
> http://listcultures.org/mailman/listinfo/p2presearch_listcultures.org
>
> Updates: http://del.icio.us/mbauwens; http://friendfeed.com/mbauwens;
> http://twitter.com/mbauwens; http://www.facebook.com/mbauwens
>
>
>
>
>
> _______________________________________________
> p2presearch mailing list
> p2presearch at listcultures.org
> http://listcultures.org/mailman/listinfo/p2presearch_listcultures.org
>
>

-- 
Ryan Lanham
rlanham1963 at gmail.com
Facebook: Ryan_Lanham
P.O. Box 633
Grand Cayman, KY1-1303
Cayman Islands
(345) 916-1712
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listcultures.org/pipermail/p2presearch_listcultures.org/attachments/20100207/0676f073/attachment-0001.html>