SYST 664 / CSI 674

Bayesian Inference and Decision Theory

Kathryn Blackmond Laskey
Department of Systems Engineering and Operations Research
George Mason University

Spring, 2016
TASC Chantilly and Online
Wednesday, 4:30-7:10 PM



Bayesian decision theory provides a unified and intuitively appealing approach to drawing inferences from observations and making rational, informed decisions.   Bayesians view statistical inference as a problem in belief dynamics, using of evidence about a phenomenon to revise and update knowledge about it. Bayesian statistics is a scientifically justifiable way to integrate informed expert judgment with empirical data.  For a Bayesian, statistical inference cannot be treated entirely independently of the context of the decisions that will be made on the basis of the inferences.  This course introduces students to Bayesian theory and modern computational methods for Bayesian inference.  Students will learn the commonalities and differences between the Bayesian and frequentist approaches to statistical inference, how to approach a statistics problem from the Bayesian perspective, and how to combine data with informed expert judgment in a sound way to derive useful and policy relevant conclusions.  Students will learn the necessary theory to develop a firm understanding of when and how to apply Bayesian and frequentist methods, and will also learn practical procedures for developing statistical models for phenomena, drawing inferences, and evaluating evidential support for hypotheses.  The course covers fundamentals of the Bayesian theory of inference, including probability as a representation for degrees of belief, the likelihood principle, the use of Bayes Rule to revise beliefs based on evidence, conjugate prior distributions for common statistical models, Markov Chain Monte Carlo methods for approximating the posterior distribution, Bayesian hierarchical models, and other key topics.  Graphical models are introduced for representing complex probability and decision problems by specifying them in modular components.  Assignments make use of modern computational techniques and focus on appling the methods to practical problems.

Syllabus

Delivery Mode

The class will be taught in-person to TASC employees at the TASC facility at 4:30PM on Wednesdays. Other students will participate remotely. Interested students can gather on the Fairfax campus at class time to participate remotely in real time.  For January 20, this room will be ENGR 2608.  The location for future weeks will be announced later.   All classes will be recorded so you can listen and review at your convenience.

Lecture Notes

Lecture notes for each unit will be made available before the first class of the unit.  This class was last offered in Spring 2011.  Links to the previous notes are provided for those who like to read ahead. There will be be changes both in the order of topics covered and the notes themselves.

Homework Assignments

Homework is due at class time on the assigned due date. If it is submitted before 23:59 the day after the due date, you will receive 75% credit.  If it is submitted up to 1 week late, you will receive 50% credit.  If you have extenuating circumstances, please contact me in advance, and I will consider giving you additional time to complete the assignment for partial credit. There is a Blackboard dropbox for electronic assignments. Assignments will be posted here and on Blackboard.

Exams

Study Aids

Sites of Possible Interest

A recent web search on "Bayesian" yielded over 12 million hits (up from 5 million in 2009).  When you have some time, it would be a good idea to throw in a modifier or two to reflect your individual interests and browse.  There is lots of very interesting stuff out there.  I've culled a few tidbits to get you started.  Many of these sites also contain useful links to other Bayesian sites.
Professional Societies:
International Society for Bayesian Analysis
American Statistical Association Section on Bayesian Statistical Science
A Non-Comprehensive Sampling of Research Groups and Stat Departments with Bayesian Orientation:
Carnegie Mellon University Department of Statistics
Duke University Department of Statistical Science
University of Washington Statistics Department
Microsoft Machine Learning and Applied Statistics Group
A Non-Comprehensive List of Free Bayesian Statistical Software:
MCMCpack (an R package for Bayesian analysis)
Bayesian Analysis Using Gibbs Sampling (BUGS)

Just Another Gibbs Sampler (JAGS)
FirstBayes
Companion software to Peter Hoff text
Companion software to Peter Lee text
Kevin Murphy's Bayesian Network Toolbox for MATLAB
UnBBayes open source plugin framework for  Probabilistic Graphical Models
Miscellaneous
The Bayesian Songbook (includes Frequentist Frenzy by world renowned songwriter Kathryn Laskey)
Bayesians Worldwide