From: Amara Graps (amara@amara.com)
Date: Sat Jan 30 1999 - 11:23:37 MST
>Michael Nielsen (mnielsen@theory.caltech.edu)
>Fri, 29 Jan 1999 11:57:51 -0800 (PST)
>On 29 Jan 1999, Anders Sandberg wrote:
>
>> Probability Theory: the Logic of Science bt E. T. Jaynes
>>
>> Fulltext (but missing certain pieces) at
>> http://omega.albany.edu:8008/JaynesBook
>
>Sadly, Jaynes died a year or so ago, so this book will never be completed.
>
>Michael
-------------------
Hello Extropes,
To follow up on the Bayes/Jaynes discussion.
E.T Jaynes died last April '98, but his "spirit" is alive and
thriving in the Maximum Entropy statistics field.
One of his old students (and a smart Bayesian scientist), Larry
Bretthorst, is doing his best to complete Jaynes' book. He still has
handwritten manuscripts from Jaynes that he is working to convert to
a publishable form. As of last summer, Bretthorst was still deciding
whether to complete the book with him (Bretthorst) filling in the
remaining chapters with his own words, or to cut out those
incomplete chapters. I believe that a book will be completed though,
in the not-too-distant future, with mostly Jaynes' words.
I was fortunate last summer to attend the Eighteenth International
Workshop on Maximum Entropy and Bayesian Methods (MaxEnt '98). It
was one in a series of "MaxEnt" conferences that have been held
every year since 1981 at different locations all over the world.
(See: http://www.ipp.mpg.de/OP/maxent98/me98.html for the 1998
conference) The general scope of the annual conferences are the
applications of the maximum entropy and Bayesian methods for diverse
areas of scientific research. The workshop last year was dedicated
to (and a special focus on) Edwin Jaynes. Because the 1998 meeting
was a special meeting devoted to Edwin Jaynes, and because many of
the meeting participants had a personal and professional
relationship with Jaynes that lasted for decades, the presentations
and coffee-break discussions provided a number of historical
summaries that gave a nice historical framework for the conference
newcomer and nice rememberences for the conference oldcomer.
I'm still a newcomer to the Bayesian Probability Theory field. I
started reading the literature bit-by-bit last Fall, but I haven't
learned enough to apply it to my own work yet. However, I've
accumulated some really nice references since last Fall that I can
share with you and written some text about the general ideas for my
colleagues.
Here, let me give an overview of Bayesian Probability Theory. I'm an
astronomer, so what follows will have an astronomy slant.
Overview
Bayesian Probability Theory is a rigorous mathematical theory
constructed from a simple notion that "probability is a measure of
degree of a proposition's plausibility."
The Bayesian approach to scientific inference takes into account not
only the raw data, but also the prior knowledge that one has to
supplement the data. That prior knowledge may be data or results
from previous experiments, conservation laws or models, known
characteristics of the assumed model, data filters, scientific
conjecture, experience, or other objective or subjective data
sources. The Bayesian approach assigns probabilities to all possible
theories and to all possible evidence. Using a logical framework for
prior and current information, the Bayesians infer a probabilistic
answer to a well-posed question, using all of the information at
one's disposal. And when one acquires new evidence, the Bayesians
update their "priors" in the equation, resulting in a modified
probabilistic answer that essentially reduces one's hypothesis
space. Probability to the Bayesians represents a state of knowledge,
conditional to some context.
The Bayesian probabilistic ideas have been around since the 1700s.
Bernoulli, in 1713, recognized the distinction between two
definitions of probability: (1) probability as a measure of the
plausibility of an event with incomplete knowlege, and (2)
probability as the long-run frequency of occurrence of an event in a
sequence of repeated (sometimes hypothetical) experiments. The
former (1) is a general definition of probability adopted by the
Bayesians. The latter (2) is called the "frequentist" view,
sometimes called the "classical", "orthodox" or "sampling theory"
view.
Astronomers who rely on frequentist definitions, while assigning
their uncertainties for their measurements, should be wary. The
concept of sampling theory, or the statistical ensemble, in
astronomy is often not relevant. For example, a gamma-ray burst is
a unique event, observed once, and the astronomer needs to know what
uncertainty to place on the one data set he/she actually has, not on
thousands of other hypothetical gamma-ray burst events. And
similarly, the astronomer who needs to assign uncertainty to the
large-scale structure of the Universe needs to assign uncertainties
based on _our_ particular Universe, because there are not similar
Observations in each of the "thousands of universes like our own."
>From my readings so far, I have found Bayesian Probability Theory to
be a consistent, logical, elegant, probabilistic framework with
which to approach and calculate answers to scientific problems. I
believe that most scientists would find that they can better
formulate the solution of their scientific problems after being
introduced to Bayesian methods, and they would derive a more
methodical (and realistic) uncertainty to their results.
Bayesian Statistics Books and Papers
This section I list several sites that I have found extremely
helpful for providing Bayesian statistics books and papers on the
Internet.
Tom Loredo is an astronomer at Cornell, who is using Bayesian
methods in his astronomy work. His persuasive papers can be found
at:
http://astrosun.tn.cornell.edu/staff/loredo/bayes/tjl.html
where one can download gzipped, postscript versions. I think any
astronomer seeking to begin to learn about Bayesian methods can do
no better than starting with his seminal "From Laplace to Supernova
SN 1987A" 60 page article, and then working through more of his
papers, for example: "The Promise of Bayesian Inference for
Astrophysics" and then "The Return of the Prodigal: Bayesian
Inference in Astrophysics".
Edwin Jaynes was one of the founders of modern-day Bayesian ideas
who died earlier last year. He had written 2/3 of a book before he
died: _Probability Theory as Extended Logic_, that one of his former
students: Larry Bretthorst is making available on the Internet,
available at:
This site would appeal to any scientist, not only astronomers,
because of the breadth and scope of Jaynes' ideas. Bretthorst's
Washington University Web site also provides his own important book:
_Bayesian Spectrum Analysis and Parameter Estimation_, as well as
several other articles by himself, Jaynes, and other Bayesians.
Giulio D'Agnostini is a statistician from Rome, who is teaching
statistics to high energy physicists at CERN. Last summer he
completed teaching a course about Bayesian statistics, and you can
find his 200 page book of detailed lecture material at a CERN site:
http://www.cern.ch/Training/ACAD/reglec_E.html
(Note: Scroll down the page to
"Bayesian Reasoning in High-Energy Physics - Principles and
Applications" by G. D'Agostini, INFN, Italy on 25, 26, 27, 28 & 29
May 1998.)
To read his lecture notes, you will need to download each gzipped,
postscript part: 0) Introduction, 1) Part 1, 2) Part 2, 2) Part 3,
3) Bibliography. D'Agnostini has a lively way of presenting his
material (if you have an opportunity to see his lectures in person,
don't miss it) and his material is similarly humorous and
interesting.
William Press, of the Harvard-Smithsonian Center for Astrophysics,
is one of the authors of the influential work: _Numerical Recipes_.
I was not aware of Press' interest in Bayesian methods until I saw a
reference in one of Tom Loredo's papers (above). Press has a very
interesting article: "Understanding Data Better with Bayesian and
Global Statistical Methods" that I located on the Los Alamos
National Laboratory's astro-ph server:
http://xxx.lanl.gov/list/astro-ph/9604?100
(and then scroll down to article: 9604126). This paper is a 14 page
postscript paper that includes embedded figures, given at the
Unsolved Problems in Astrophysics Conference, Princeton, April 1995.
And for some code:
http://www.astro.psu.edu/statcodes/
StatCodes is a metasite of over 200 links of on-line statistical
software (in Fortran, C, executable binaries, others) for astronomy
and related fields from the astrostatistics research group at Penn
State, who also operate the Statistical Consulting Center for
Astronomy. This site lists links to free source code, available on
the Internet, from small, as well as large (for example, from
StatLib at Carnegie Mellon University) software libraries.
This site is nicely organized in terms of topic from general
statistics packages and information to Bayesian statistics to time
series analysis to density estimation and smoothing to correlation and
regression to visualization software and interactive Web software. You
can search the StatCodes site by keywords, as well.
And some Non-Internet reference books:
Sivia, D.S. _Data Analysis: A Bayesian Tutorial_, Clarendon Press:
Oxford, 1996.
Martz, Harry and Waller, Ray, chapter: "Bayesian Methods" in
_Statistical Methods for Physical Science_, Editors: John L.
Stanford and Stephen Vardeman [Volume 28 of the Methods of
Experimental Physics], Academic Press, 1994, pg. 403-432.
Amara Graps
***************************************************************
Amara Graps | Max-Planck-Institut fuer Kernphysik
Interplanetary Dust Group | Saupfercheckweg 1
+49-6221-516-543 | 69117 Heidelberg, GERMANY
Amara.Graps@mpi-hd.mpg.de * http://galileo.mpi-hd.mpg.de/~graps
***************************************************************
"Never fight an inanimate object." - P. J. O'Rourke
This archive was generated by hypermail 2.1.5 : Fri Nov 01 2002 - 15:02:58 MST