[p2p-research] Applying utility functions to humans considered harmful

Thu Feb 4 03:20:45 CET 2010

the pdf won't load here, but that seems to be the Van Gelder I was referring
to in a discussion with jandrews, as saying that equating brains and humans
as machines belonged to the infantile phase of AI,

Michel

On Thu, Feb 4, 2010 at 8:21 AM, Ryan <rlanham1963 at gmail.com> wrote:

> For the utilitarians out there...
>
>
>
> Sent to you by Ryan via Google Reader:
>
>
> Applying utility functions to humans considered harmful<http://lesswrong.com/lw/1qk/applying_utility_functions_to_humans_considered/>
> via lesswrong: What's new <http://lesswrong.com/> on 2/3/10
>
> Submitted by Kaj_Sotala <http://lesswrong.com/user/Kaj_Sotala> 19 comments<http://lesswrong.com/lw/1qk/applying_utility_functions_to_humans_considered/#comments>
>
> There's a lot of discussion on this site that seems to be assuming
> (implicitly or explicitly) that it's meaningful to talk about the utility
> functions of individual humans. I would like to question this assumption.
>
> To clarify: I don't question that you couldn't, *in principle*, model* *a
> human's preferences by building this insanely complex utility function. But
> there's an infinite amount of methods by which you could model a human's
> preferences. The question is which model is the most useful, and which
> models have the least underlying assumptions that will lead your intuitions
> astray.
>
> Utility functions are a good model to use if we're talking about designing
> an AI. We want an AI to be predictable, to have stable preferences, and do
> what we want. It is also a good tool for building agents that are immune to
> Dutch book tricks. Utility functions are a bad model for beings that do not
> resemble these criteria.
>
> To quote Van Gelder (1995)<http://people.bu.edu/pbokulic/class/vanGelder-reading.pdf>
> :
>
> Much of the work within the classical framework is mathematically elegant
> and provides a useful description of optimal reasoning strategies. As an
> account of the actual decisions people reach, however, classical utility
> theory is seriously flawed; human subjects typically deviate from its
> recommendations in a variety of ways. As a result, many theories
> incorporating variations on the classical core have been developed,
> typically relaxing certain of its standard assumptions, with varying degrees
> of success in matching actual human choice behavior.
>
> Nevertheless, virtually all such theories remain subject to some further
> drawbacks:
>
> (1) They do not incorporate any account of the underlying motivations that
> give rise to the utility that an object or outcome holds at a given time.
> (2) They conceive of the utilities themselves as static values, and can
> offer no good account of how and why they might change over time, and why
> preferences are often inconsistent and inconstant.
> (3) They offer no serious account of the deliberation process, with its
> attendant vacillations, inconsistencies, and distress; and they have nothing
> to say about the relationships that have been uncovered between time spent
> deliberating and the choices eventually made.
>
> Curiously, these drawbacks appear to have a common theme; they all concern,
> one way or another, *temporal* aspects of decision making. It is worth
> asking whether they arise because of some deep structural feature inherent
> in the whole framework which conceptualizes decision-making behavior in
> terms of calculating expected utilities.
>
> One model that attempts to capture actual human decision making better is
> called *decision field theory*. (I'm no expert on this theory, having
> encountered it two days ago, so I can't vouch for how good it actually is.
> Still, even if it's flawed, it's useful for getting us to think about human
> preferences in what seems to be a more realistic way.) Here's a brief
> summary of how it's constructed from traditional utility theory, based on Busemeyer
> & Townsend (1993) <http://mypage.iu.edu/%7Ejbusemey/psy_rev_1993.pdf>. See
> the article for the mathematical details, closer justifications and
> different failures of classical rationality which the different stages
> explain.
>
> *Stage 1: Deterministic Subjective Expected Utility (SEU) theory.*Basically classical utility theory. Suppose you can choose between two
> different alternatives, A and B. If you choose A, there is a payoff of 200
> utilons with probability S1, and a payoff of -200 utilons with probability
> S2. If you choose B, the payoffs are -500 utilons with probability S1 and
> +500 utilons with probability S2. You'll choose A if the expected utility of
> A, S1 * 200 + S2 * -200 is higher than the expected utility of B, S1 * -500
> + S2 * 500, and B otherwise.
>
> *Stage 2: Random SEU theory. *In stage 1, we assumed that the
> probabilities S1 and S2 stay constant across many trials. Now, we assume
> that sometimes the decision maker might focus on S1, producing a preference
> for action A. On other trials, the decision maker might focus on S2,
> producing a preference for action B. According to random SEU theory, the
> attention weight for variable S*i* is a continous random variable, which
> can change from trial to trial because of attentional fluctuations. Thus,
> the SEU for each action is also a random variable, called the *valence* of
> an action. Deterministic SEU is a special case of random SEU, one where the
> trial-by-trial fluctuation of valence is zero.
>
> *Stage 3: Sequential SEU theory.* In stage 2, we assumed that one's
> decision was based on just one sample of a valence difference on any trial.
> Now, we allow a sequence of one or more samples to be accumulated during the
> deliberation period of a trial. The attention of the decision maker shifts
> between different anticipated payoffs, accumulating weight to the different
> actions. Once the weight of one of the actions reaches some critical
> threshold, that action is chosen. Random SEU theory is a special case of
> sequential SEU theory, where the amount of trials is one.
>
> Consider a scenario where you're trying to make a very difficult, but very
> important decisions. In that case, your inhibitory threshold for any of the
> actions is very high, so you spend a lot of time considering the different
> consequences of the decision before finally arriving to the (hopefully)
> correct decision. For less important decisions, your inhibitory threshold is
> much lower, so you pick one of the choices without giving it too much
> thought.
>
> *Stage 4: Random Walk SEU theory. *In stage 3, we assumed that we begin to
> consider each decision from a neutral point, without any of the actions
> being the preferred one. Now, we allow prior knowledge or experiences to
> bias the initial state. The decision maker may recall previous preference
> states, that are influenced in the direction of the mean difference.
> Sequential SEU theory is a special case of random walk theory, where the
> initial bias is zero.
>
> Under this model, decisions favoring the status quo tend to be chosen more
> frequently under a short time limit (low threshold), but a superior decision
> is more likely to be chosen as the threshold grows. Also, if previous
> outcomes have already biased decision A very strongly over B, then the mean
> time to choose A will be short while the mean time to choose B will be long.
>
> *Stage 5: Linear System SEU theory. *In stage 4, we assumed that previous
> experiences all contribute equally. Now, we allow the impact of a valence
> difference to vary depending on whether it occurred early or late (a primacy
> or recency effect <http://en.wikipedia.org/wiki/Serial_position_effect>).
> Each previous experience is given a weight given by a growth-decay rate
> parameter. Random walk SEU theory is a special case of linear system SEU
> theory, where the growth-decay rate is set to zero.
>
> *Stage 6: Approach-Avoidance Theory. *In stage 5, we assumed that, for
> example, the average amount of attention given to the payoff (+500) only
> depended on event S2. Now, we allow the average weight to be affected by a
> another variable, called the goal gradient. The basic idea is that the
> attractiveness of a reward or the aversiveness of a punishment is a
> decreasing function of distance from the point of commitment to an action.
> If there is little or no possibility of taking an action, its consequences
> are ignored; as the possibility of taking an action increases, the attention
> to its consequences increases as well. Linear system theory is a special
> case of approach-avoidance theory, where the goal gradient parameter is
> zero.
>
> There are two different goal gradients, one for gains and rewards and one
> for losses or punishments. Empirical research suggests that the gradient for
> rewards tends to be flatter than that for punishments. One of the original
> features of approach-avoidance theory was the distinction between rewards
> versus punishments, closely corresponding to the distinction of positively
> versus negatively framed outcomes made by more recent decision theorists.
>
> *Stage 7: Decision Field Theory. *In stage 6, we assumed that the time
> taken to process each sampling is the same. Now, we allow this to change by
> introducing into the theory a time unit *h*, representing the amount of
> time it takes to retrieve and process one pair of anticipated consequences
> before shifting attention to another pair of consequences. If *h* is
> allowed to approach zero in the limit, the preference state evolves in an
> approximately continous manner over time. Approach-avoidance is a spe... you
> get the picture.
>
>
> ------------------------------
>
>
>
> Now, you could argue that all of the steps above are just artifacts of
> being a bounded agent without enough computational resources to calculate
> all the utilities precisely. And you'd be right. And maybe it's meaningful
> to talk about the "utility function of humanity" as the outcome that occurs
> when a CEV-like entity calculated what we'd decide if we could collapse
> Decision Field Theory back into Deterministic SEU Theory. Or maybe you just
> say that all of this is low-level mechanical stuff that gets included in the
> "probability of outcome" computation of classical decision theory. But which
> approach do you think gives us more useful conceptual tools in talking about
> modern-day humans?
>
> You'll also note that even DFT (or at least the version of it summarized in
> a 1993 article) assumes that the payoffs themselves do not change over time.
> Attentional considerations might lead us to attach a low value to some
> outcome, but if we were to actually end up in that outcome, we'd always
> value it the same amount. This we know to be untrue. There's probably some
> even better way of looking at human decision making, one which I suspect
> might be very different from classical decision theory.
>
> So be extra careful when you try to apply the concept of a utility function
> to human beings.
>
>
>
> Things you can do from here:
>
>    - Subscribe to lesswrong: What's new<http://www.google.com/reader/view/feed%2Fhttp%3A%2F%2Flesswrong.com%2F.rss?source=email>using
>    *Google Reader*
>    - Get started using Google Reader<http://www.google.com/reader/?source=email>to easily keep up with
>    *all your favorite sites*
>
>
>
>
> _______________________________________________
> p2presearch mailing list
> p2presearch at listcultures.org
> http://listcultures.org/mailman/listinfo/p2presearch_listcultures.org
>
>

-- 
Work: http://en.wikipedia.org/wiki/Dhurakij_Pundit_University - Think thank:
http://www.asianforesightinstitute.org/index.php/eng/The-AFI

P2P Foundation: http://p2pfoundation.net  - http://blog.p2pfoundation.net

Connect: http://p2pfoundation.ning.com; Discuss:
http://listcultures.org/mailman/listinfo/p2presearch_listcultures.org

Updates: http://del.icio.us/mbauwens; http://friendfeed.com/mbauwens;
http://twitter.com/mbauwens; http://www.facebook.com/mbauwens
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listcultures.org/pipermail/p2presearch_listcultures.org/attachments/20100204/6a24f0b0/attachment-0001.html>