Bayesian Inference for the Physical Sciences

Rev. Thomas Bayes (1702-1761) and Pierre Simon Laplace (1749-1827)

Penn State's Center for Astrostatistics and SAMSI will jointly host a winter school in astrostatistics 18-25 January 2006, including 3 days devoted to Bayesian methods and 2 days devoted to nonparametric and machine learning methods. The school also includes 2 days of tutorials on astronomy for statisticians. See the CASt School page for info.

Welcome to BIPS: Bayesian Inference for the Physical Sciences, an annotated online index/clearinghouse for information on the Bayesian approach to statistical inference of special relevance to applications in the physical sciences.

This page is constantly evolving, so I hope you'll visit regularly. To guide you in your repeat visits, the What's New AT BIPS page summarizes recent changes in reverse chronological order.

If you come across an online resource that you think should be referenced here, please don't hesitate to pass it along to the BIPS webslave, via the links at the bottom of this and other pages.

Bayes's Theorem


General Texts and Tutorials

Probability Theory: The Logic of Science
This site hosts PDF and PostScript files of physicist Ed Jaynes's monumental treatise on Bayesian inference, the first volume of which will be published in 1999/2000 by Cambridge University Press. The same site hosts an excellent brief biography of Ed Jaynes written by his last graduate student, Larry Bretthorst.

Jaynes: Probability Theory---The Logic of Science
Yet another site archiving the chapters of Jaynes's book in PostScript, made available here by mathematician Carlos Rodriguez. The version here is not the latest version.

Bayesian Spectrum Analysis and Parameter Estimation
This 1988 book by Larry Bretthorst is a very readable and practical introduction to Bayesian inference applied to the analysis of time series data with additive Gaussian noise. Published by Springer-Verlag, it is now out of print, but by arrangement with the publishers is available online at the Washington University Bayesian Reprints web site maintained by Larry.

Washington University Bayesian Reprints
This site archives tutorial and application papers by Ed Jaynes, Larry Bretthorst, Stephen Gull, David Mackay, Devinder Sivia, Phil Gregory, and Tom Loredo (physical scientists associated with the MaxEnt conferences). The tutorial papers include the following (PDF links provided here; PostScript links also available via the URL above):

Data Analysis: A Bayesian Tutorial
This is an undergraduate text on Bayesian inference written for physical scientists by Devinder Sivia, and published by Oxford University Press. It is available in paperback. A bit more info, including some readers' comments, can be found at Sivia's home page. Originally published in Summer of 1996, this was the first "modern" Bayesian book written expressly for physical scientists, and built on the work of the leading physicist exponents of Bayesian methods (Jaynes, Bretthorst, Gull, and Skilling). Please eschew the strange use of the term "evidence" in this otherwise fine book; this usage is unique to the Cambridge/Oxford MaxEnt practitioners, and the term is used differently in other Bayesian literature. A revision is in progress that will update the book with material on modern computational techniques.

Bayesian Logical Data Analysis for the Physical Sciences: A Comparative Approach with Mathematica Support
This is the newest book (May 2005) on Bayesian methods for physical scientists, written by astronomer Phil Gregory. Aimed at graduate students, it covers the fundamentals at a level between that of the Jaynes and Sivia books. It has a much more "applied" focus than Jaynes's book, and covers several applications at a level well beyond what is addressed in Sivia's book. There is particularly extensive coverage of spectral analysis (detecting and measuring periodic signals), including a self-contained introduction to Fourier methods. It also includes some treatment of frequentist methods. It is currently the only book written for physical scientists that covers modern Bayesian computational methods (i.e., MCMC), though only a small part of the "computational landscape" is explored. It is available in hardcover and in eBook format. A Mathematica notebook with material supporting the book is available for download from the BLDA support page. Gregory has actively taught Bayesian methods to physics and astronomy students for over a decade; the book certainly benefits from his extensive classroom experience.

Bayesian Reasoning in High Energy Physics: Principles and Applications
Giulio D'Agostini, an outspoken proponent of Bayesian methods in high energy physics, has extensive course notes on basic Bayesian statistics archived at the CERN Lecture Program web site (see the 1997/1998 section) as a series of 5 PostScript files with over 150 pages: This report is also available as CERN "Yellow" Report CERN-99-03.

Maximum Entropy Data Consultants: References
Included on this page is John Skilling's article from the Journal of Microscopy (1998), "Probabilistic Data Analysis; An Introductory Guide." This gives a brief introduction to the Bayesian approach, and an introductory overview of various Monte Carlo methods for doing Bayesian calculations. Please eschew the strange use of the term "evidence" in this otherwise fine review; this usage is unique to the Cambridge/Oxford MaxEnt practitioners, and the term is used differently in other Bayesian literature.

Kenneth Hanson's Home Page
Hanson worked at the Los Alamos National Lab using Bayesian methods to deal with problems in uncertainty quantification for physics models. This site includes articles documenting the work of Hanson and his collaborators, as well as tutorial articles by Hanson and by guest speakers visiting his Uncertainty Quantification Working Group. These include:

Tom Loredo's Bayesian Reprints
Tutorial and research papers by the editor of the BIPS web site. Three general tutorials and reviews that may be of particular interest to BIPS visitors include: Visit the link above to find descriptions and links for these papers.


Bayesian Analysis E-print Archive
Hosted by the Los Alamos XXX server that hosts the main reprint archives in physics and astronomy, this part of the archive sees little traffic, but the papers that do appear here are usually of specific relevance to physical scientists.

ISBA/SBSS Bayesian Abstract Archive
An archive of Bayesian publications, including technical reports and theses as well as more publicly available publications, hosted by ISDS at Duke and sponsored by ISBA (International Society for Bayesian Analysis) and SBSS (Section on Bayesian Statistical Sciences of the American Statistical Association).

MCMC Preprints
A well-maintained collection of pointers and bibliographic information for publications on Markov Chain Monte Carlo (MCMC) methods.

Perfectly Random Sampling from Markov Chains
David Wilson's web page providing annotated bibliographic information for articles describing a new (1996) approach to using Markov Chains to simulate distributions arising in statistical mechanics, Bayesian inference, and other fields. This approach, "perfectly random sampling" or "exact sampling," removes the problem of assessing convergence of Markov chains (for some problems), and can be used to create a set of iid samples from the Markov chain's stationary distribution. A brief and basic tutorial on exact sampling is available at Radford Neal's Miscellaneous Documents page.

Washington University Bayesian Reprints
This site archives tutorial and application papers by Ed Jaynes, Larry Bretthorst, David Mackay, and Tom Loredo (physical scientists associated with the MaxEnt conferences).

Bayesian Model Averaging Home Page
Chris Volinsky's collection of links to articles and software on Bayesian model averaging, a technique for accounting for uncertainty in model specification when making inferences.

Space Telescope Science Institute Search Page
STSci archives a number of papers in HTML format describing Bayesian methods for "inverting" complicated imaging data. You can locate these via the search facility linked here (try searching for "bayesian").

G. D'Agostini - Probability and Statistics
D'Agostini, a leading proponent of Bayesian methods in particle physics, collects here many of his papers on the Bayesian approach.

Geophysical Inverse Theory
John Scale's web site with extensive course notes, a review paper, and other papers on Bayesian approaches to solving geophysical inverse problems, including software resources in Xlist-Stat and Mathematica.

Maximum Entropy Data Consultants: References
This page from the MEDC web site collects a few of the references by John Skilling and Sibusiso Sibisi on the "Massive Inference" approach to Bayesian density estimation (using infinitely divisible process models, such as gamma, Dirichlet, and compound Poisson processes). Further references on this topic are available via ftp from DAMTP at U. Cambridge.

Non-Subjective Bayesian Statistical Methodology
This site maintains a PostScript catalog of "noninformative" priors of various types for various problems (conventional priors, reference priors, etc.), as well as lists of conferences and researchers concerned with "default" Bayesian methods.

Tom Loredo's Bayesian Reprints
Tutorial and research papers by the editor of the BIPS web site.

Bayesian Software

Note: (P) indicates a package with multiple functions, documentation, etc. Other programs are often single-purpose subroutines with little on-line documentation.
StatLib (CMU)
StatLib, hosted by Carnegie Mellon University, is a system for distributing statistics software, much of it Bayesian.

The AutoClass Project
Unsupervised Bayesian classification system that seeks a maximum posterior probability classification for multivariate data, including mixed real-value and discrete data with missing values. (P)

BAYESPACK, etc. (Alan Genz's Home Page)
Genz is a leader in the development of new algorithms for numerical computation of mulitiple integrals; his recent research focuses on integrals that arise in Bayesian inference. His homepage provides many of his papers in PostScript form, and collections of FORTRAN software implementing and demonstrating his algorithms. His BAYESPACK collection will be of particular interest to those doing Bayesian calculations of modest (up to 10 or so) dimensions. SunOS Unix users may prefer the gzipped tar file version of the package here at BIPS to Genz's piecemeal distribution (the tar version also fixes a minor incompatibility with the SunOS f77 compiler). (P)

BUGS: Bayesian inference Using Gibbs Sampling (also an ftp directory)
BUGS is a program for Bayesian inference using the Gibbs Sampler Markov chain Monte Carlo technique produced by the Biostatistics Unit of the Medical Research Council of the United Kingdom. It is written in Modula 2 and distributed as compiled code for a variety of platforms. The sites above host the software and documentation in dowloadable files. There is also an extensive online HTML manual for BUGS. (P)

Radford Neal's software for flexible Bayesian modeling
"This software supports Bayesian regression and classification models based on neural networks and Gaussian processes, and Bayesian mixture models. It also supports a variety of Markov chain sampling methods, which may be applied to distributions specified by simple formulas, including simple Bayesian models defined by formulas for the prior and likelihood. The latest version, of 1999-03-13, allows you to try lots of Markov chain sampling methods on Bayesian models defined using a BUGS-like notation." The software consists of a suite of command-line programs that can be chained together (interacting through data stored in files) to make inferences. (ANSI C for Unix, with support for xgraph under X-windows.) (P)

Bayesian Model Averaging Home Page
Chris Volinsky's collection of links to articles and software on Bayesian model averaging, a technique for accounting for uncertainty in model specification when making inferences. Includes links to several S-PLUS packages.

Gibbs sampler iterations
Calculates the number of iterations needed in a Markov chain Monte Carlo run.

First Bayes
Teaching package for elementary Bayesian statistics. (P)

Belief Networks
Links to software for graphical belief functions, Bayesian networks and influence diagrams. (P)

Bayes linear methods based on expectation and covariance structures.

Math 648 - Bayesian Inference
Minitab examples.

MatLab Scripts for Bayesian Blocks
Jeff Scargle here provides the text of his 1998 Astrophysical Journal article on Bayesian blocks (a Poisson changepoint model for detecting variability) and MatLab scripts and sample data for doing Bayes Blocks calculations.

Bayesian Time Series Analysis Software by Mike West and Colleagues
This site provides pointers to three collections of software Mike West (ISDS) and his colleagues have produced, as well as collections of data sets for software testing and development. Included are the BATS software (DOS/Win) for the book Applied Bayesian Forecasting and Time Series Analysis, and Fortran90/S-plus software for nonstationary time series analysis and analysis with autoregressive component models. (P)

Maximum Entropy Data Consultants
This commercial firm based in the UK was founded by astronomers John Skilling and Steve Gull, and sells the influential and widely used MEMSYS code for quantified (i.e., Bayesian) maximum entropy "deconvolution" of data. They also develop software implementing "massive inference" techniques for Bayesian density estimation.

Miscellaneous Bayesian Resources


ISBA: International Society for Bayesian Analysis
The main international organization promoting the development and application of Bayesian methods, at There is also still some ISBA info at the old ISBA site at Albany.

ASA Section on Bayesian Statistical Sciences

Bayesians Worldwide: Bayesian Statistics Personal Web Pages

Home Pages of Bayesian Groups/Centers:

ISDS: Institute for Statistics and Decision Sciences
Based at Duke University in North Carolina, ISDS hosts a graduate program in statistics that includes a strong focus on Bayesian inference.

Bayesian Model-Based Learning Group
Located at NASA Ames Research Center, this group works on the theory and associated algorithms for various kinds of general data analysis techniques using Bayesian inference. The AutoClass home page is here, as well as a site on Bayesian search methods.

Cambridge University Inferential Sciences Group (also here)
This is the home page for the Cambridge Bayesian Inference/MaxEnt group. They are perhaps most closely associated with Maximum Entropy methods, but in the 90s their work has taken on a more general Bayesian flavor.

Harvard/CfA Astronomy & Statistics Working Group
This group consists of astronomers from the Harvard-Smithsonian Center for Astrophysics and statisticians from Harvard University's Department of Statistics, collaborating on astrostatistics problems. Their emphasis is on using hierarchical Bayesian models and MCMC algorithms to analyze data from the Chandra X-ray satellite. Some of their methodology is available in the Chandra Interactive Analysis of Observations (CIAO) software.

MPIPP Bayesian Data Analysis Group
This is the home page for a group of scientists at the Max Planck Institute for Plasma Physics who develop Bayesian methodology and software for various general data analysis problems. They hosted the 1998 Workshop on Maximum Entropy and Bayesian Methods. Their interests include inverse problems, wavelets, and signal and background separation.

Albany MaxEnt Page
This small site has links with information about the recent annual workshops on Maximum Entropy and Bayesian methods.

Non-Subjective Bayesian Statistical Methodology
This site hosts lists of publications, reports, conferences, and contacts regarding the use of "default" priors in Bayesian inference.

Dept. of Statistics at Carnegie Mellon University
Though not a Bayesian group, with Department members like Jay Kadane, Rob Kass, and Larry Wasserman, this group produces a lot of Bayesian work. They are especially interested in interdisciplinary applications, and run an annual workshop on applied Bayesian statistics. CMU also hosts the StatLib statistics software archive.

Statistics at Univeristy of Washington
A site with notable Bayesian content from Adrian Raftery, Julian Besag, Peter Hoff, Matthew Stephens and their colleagues on such topics as spatial statistics, nonparametrics, model averaging, clustering, and classification.

Conference Information:

MaxEnt 2000: 20th International Workshop on Maximum Entropy and Bayesian Methods (alternate URL)
This year's workshop will be hosted by Ali Mohammad-Djafari in France, 8-13 July 2000.

6th Valencia International Meeting on Bayesian Statistics
The major regular gathering of Bayesian statisticians, held every three to four years in Spain. The 6th meeting will be held from June 6 to June 10, 1998.

Fourth Workshop on Bayesian Statistics in Science and Technology
This annual workshop, hosted by the Statistics Department at Carnegie-Mellon University, offers a variety of lecture and poster presentations, with special in-depth lectures and discussion of a few chosen case studies of practical applications of Bayesian inference. The 4th Workshop will be held September 25-27, 1997.

Bayesian Signal Processing
This is a 1-week session (20-29 July 1998) offered as part of a six month (July to December 1998) program on Nonlinear and Nonstationary Signal Processing hosted by the Isaac Newton Institute for Mathematical Sciences in Cambridge, England. Topics to be covered include Bayesian numerical methods, nonlinear/nonstationary time series, forecasting, changepoint modeling, dynamical systems, and applications in econometrics and environmental and spatial data analysis.

The Bayesian Songbook
Every three to four years there is a large gathering of Bayesian statisticians in Valencia, Spain; these are the famous and influential "Valencia" meetings that have produced a series of conference proceedings simply titled Bayesian Statistics 5, etc.. A tradition of these meetings is a "cabaret" performance by participants, typically featuring at least one "adaptation" of a popular song with lyrics altered to refer to some aspect of Bayesian statistics. This web site collects the lyrics in LaTeX and PostScript, and even a few audio files (WAV format).

On Thomas Bayes:

MacTutor Bio



General Statistical Resources With Bayesian Content

Virtual Library Entry on Statistics

StatLib (CMU)
StatLib, hosted by Carnegie Mellon University, is a system for distributing statistics software, much of it Bayesian.

"StatCodes is a metasite with links to over 200 sites providing free on-line codes implementing statistical methods that may be useful to astronomers. Methods include time series analysis, multivariate analysis, censored and truncated data, nonparametric statistics, correlation and regression, Bayesian methods (hosted by Tom Loredo, Cornell University [the site you are at right now!]), density estimation and smoothing, image analysis, spatial statistics, visualization tools, interactive Web tools, and multipurpose statistics packages. In some fields like signal processing and wavelet analysis, the resources are vast and links are given to other metasites. Some codes are single subroutines (usually in Fortran or C), while others are full packages with documentation. The contents of StatCodes is searchable through the Astronomical Software Directory Service ( StatCodes welcomes links to new codes, particularly those written by astronomers. StatCodes is maintained by Penn State astronomer Eric Feigelson ("

The DATA Center Home Page
The Center for Data Analysis Technology and Applications is an informal forum for exchanging ideas about new data analysis technology with astrophysical applications and is hosted by Jeff Scargle at NASA/Ames. The web site hosts descriptions of talks presented at DATA Center meetings and a bibliography of publications of members. Scargle and a few other members work on astrophysical applications of Bayesian inference.

PDG: Particle Data Group
The home page for the Particle Data Group that produces the annual Review of Particle Physics which includes recommendations of statistical methods for common data analysis tasks in high energy physics. The little Bayesian content in the Review remains a subject of continuing controversy. The statistical content can be found in the "Mathematical Tools" section of the Review.

The Data Analysis BriefBook
An on-line version of a standard particle physics reference on data analysis methods, prepared and hosted by CERN. Bayesian content is virtually nonexistant, but it's a useful reference for its description of current statistical practice in high energy physics.

Thanks to Eric Feigelson for providing a head start in locating online sources of Bayesian software.

Tom Loredo's Astro Home Page /