From: Amara Graps (amara@amara.com)
Date: Wed Oct 27 1999 - 14:36:55 MDT
James Wetterau (jwjr@ignition.name.net) Tue, 26 Oct 1999 writes:
>So far, the only non-circular definition I have found is a
>mathematical definition claiming certainty in results iff we
>take limits as the number of experiments goes to infinity.
That's the "frequentist" definition of probability. I don't find the
definition satisfying either. You may find the Bayesian perspective
of probability more natural and intutive.
(Some portions of the following note I extracted from an article I
wrote 1 year ago about the Bayesians and the MaxEnt'98 conference
for Scientific Computing World (October 1998),
copyright 1998 IOP Publishing, UK.)
The Bayesian probabilistic ideas have been around since the 1700s.
Bernoulli, in 1713, recognized the distinction between two
definitions of probability: (1) probability as a measure of the
plausibility of an event with incomplete knowlege, and (2)
probability as the long-run frequency of occurrence of an event in a
sequence of repeated (sometimes hypothetical) experiments. The
former (1) is a general definition of probability adopted by the
Bayesians. The latter (2) is called the "frequentist" view,
sometimes called the "classical", "orthodox" or "sampling theory"
view.
In the sciences, I think that the concept of "sampling theory", or
the "statistical ensemble" is goofy, or simply not applicable. For
example, a gamma-ray burst is a unique event, observed once, and the
astronomer needs to know what uncertainty to place on the one data
set he/she actually has, not on thousands of other hypothetical
gamma-ray burst events. And similarly, the scientist who needs to
assign uncertainty to the large-scale structure of the Universe
needs to assign uncertainties based on _our_ particular Universe,
because there are not similar Observations in each of the "thousands
of universes like our own."
The Bayesian approach to scientific inference takes into account not
only the raw data, but also the prior knowledge that one has to
supplement the data. That prior knowledge may be data or results
from previous experiments, conservation laws or models, known
characteristics of the assumed model, data filters, scientific
conjecture, experience, or other objective or subjective data
sources. The Bayesian approach assigns probabilities to all possible
theories and to all possible evidence. Using a logical framework for
prior and current information, the Bayesians infer a probabilistic
answer to a well-posed question, using all of the information at
one's disposal. And when one acquires new evidence, the Bayesians
update their "priors" in the equation, resulting in a modified
probabilistic answer that essentially reduces one's hypothesis
space. Probability to the Bayesians represents a state of knowledge,
conditional to some context. Bayes' Theorem encapsulates the
process of learning. Bayesian Probability Theory may be the best
formal theory that we have on the relationship between theory and
evidence.
I'm doing my best to spread the word about Bayesian Probability Theory
to other astronomers. Some small percentage (say about 10%) have heard
about it. I think it would really open the door to a better way
(i.e. more realistic way )for scientists to formulate their
scientific problems.
In contrast to Bayes' Theorem, which is a simple, intuitive description
about one's state of knowledge, the assignment of the probabilities
can be difficult, so this is the usual "hangup" (I'm at this place,
for example, in formulating my own scientific problems in the Bayesian
way. I have some big learning yet.). There is nothing within the theory
that prescribes how one should assign probabilities; it must
come from outside. Bayesians will tell you that this aspect of thinking
more carefully about one's data gives one a deeper understanding of
one's scientific problem, and hence the extra thought involved in
assigning priors is well-worth the effort.
There are a handful of scientists using Bayesian Probability Theory
to clarify some fundamental scientific theories. Many scientific
fields such as information theory, quantum physics, Fourier
analysis, relativity, cosmology, and artificial intelligence are
full of paradoxes and fallacies. What lies at the source of the
confusion? The confusion may arise when one interprets probability
as a real, physical property, that is, when one has failed to see a
distinction between reality and our _knowledge of_ reality. In the
Bayesian view, the probability distributions used for inference do
not describe a real property of the world, only a certain state of
information about the world. For example, for the Bayesian, the wave
functions of the Schroedinger wave equation simply become a
posterior probability describing our incomplete information about
the quantum system, rather than wave functions that collapse in
reality upon our observation. A Bayesian probabilistic approach to
these fields offers some new chances to clarify many physical
"paradoxes".
>Oddly, probability appears to be a concept which people
>intuitively understand but which can only be non-circularly
>defined in terms of infinite series. I have to wonder if that,
>too, begs the question, as it is an appeal to a non-testable
>hypothesis. (If only we could have infinite trials, you'd
>see!) There's something very fishy about probability.
I hope that you can see now, that there are better definitions
of probability (in my opinion).
Here's where you can learn more.
The following two references are excellent introductions
to Bayesian methods:
Loredo, Tom, "From Laplace to Supernova SN 1987A: Bayesian
Inference in Astrophysics", as published in the Dartmouth
proceedings: International Society of Entropic Research. An
excellent tutorial on probability theory. A one page list of errata
is included, and an index. You may download the paper from:
ftp://bayes.wustl.edu/pub/Loredo/ (This Web site contains
many other useful and classic Bayesian papers, including the
most important reference: an unfinished book by Edwin Jaynes.)
(And you can find more of Tom Loredo's papers here:
http://astrosun.tn.cornell.edu/staff/loredo/bayes/tjl.html)
Giulio D'Agnostini is a statistician from Rome, who is teaching
statistics to high energy physicists at CERN. Two summers ago he
completed teaching a course about Bayesian statistics, and you can
find his 200 page book of detailed lecture material at a CERN site:
http://www.cern.ch/Training/ACAD/reglec_E.html
(Note: Scroll down the page to
"Bayesian Reasoning in High-Energy Physics - Principles and
Applications" by G. D'Agostini, INFN, Italy on 25, 26, 27, 28 & 29
May 1998.)
William Press, of the Harvard-Smithsonian Center for Astrophysics,
is one of the authors of the influential work: _Numerical Recipes_.
I was not aware of Press' interest in Bayesian methods until I saw a
reference in one of Tom Loredo's papers (above). Press has a very
interesting article: "Understanding Data Better with Bayesian and
Global Statistical Methods" that I located on the Los Alamos
National Laboratory's astro-ph server:
http://xxx.lanl.gov/list/astro-ph/9604?100
(and then scroll down to article: 9604126). This paper is a 14 page
postscript paper that includes embedded figures, given at the
Unsolved Problems in Astrophysics Conference, Princeton, April 1995.
And some Non-Internet reference books:
Sivia, D.S. _Data Analysis: A Bayesian Tutorial_, Clarendon Press:
Oxford, 1996.
Martz, Harry and Waller, Ray, chapter: "Bayesian Methods" in
_Statistical Methods for Physical Science_, Editors: John L.
Stanford and Stephen Vardeman [Volume 28 of the Methods of
Experimental Physics], Academic Press, 1994, pg. 403-432.
Amara
********************************************************************
Amara Graps email: amara@amara.com
Computational Physics vita: finger agraps@shell5.ba.best.com
Multiplex Answers URL: http://www.amara.com/
********************************************************************
"Trust in the Universe, but tie up your camels first."
(adaptation of a Sufi proverb)
This archive was generated by hypermail 2.1.5 : Fri Nov 01 2002 - 15:05:38 MST