Instructor Notes
This is a placeholder file. Please add content here.
Short introduction to Bayesian statistics
Instructor Note
- Prior specification is a common source of confusion.
The goal of this episode is to introduce the basic Bayesian modeling framework and build intuition for posterior inference:
- Specify a model in terms of a likelihood and prior;
- Understand the posterior as the combination of prior information and observed data;
- Summarize posterior information using point estimates, intervals, probabilities;
- Understand necessity of posterior sampling.
What to emphasize
- Bayesian inference is fundamentally about updating information: the prior describes understanding before observed the data is taken into account. The posterior combines prior understanding and data.
- The likelihood encodes the statistical model.
- The posterior is a distribution so points point estimates are always reductionist.
- Grid approximation is used as a pedagogical tool, concretizing prior, likelihood and posterior. Moreover, it illustrates the challenges necessitating sample-based analysis.
- The two-dimensional example illustrates:
- joint posterior distributions;
- marginalizing over nuisance parameters;
- the difficulty of using grid approximation for higher-dimensional models.
- Posterior samples offer a convenient representation of the posterior; point estimates, probabilities, and predictions are easily calculated from samples.
- Posterior predictions include uncertainty about model parameters and variability in future observations.
Tips
- Focus on developing intuition about prior –> likelihood –> posterior rather than on Bayes’ theorem algebra.
- The handedness example is deliberately simple. Encourage learners to experiment with the prior and sample size to observe their relative influence.
- Make the transition from grids to samples explicit: the remaining course operates on posterior samples.
Instructor Note
Actual value from a study from 1975 with 7,688 children in US grades 1-6 was 9.6%
Hardyck, C. et al. (1976), Left-handedness and cognitive deficit https://en.wikipedia.org/wiki/Handedness
Stan
Instructor Note
The goal of this episode is to teach the basic Stan workflow:
- Implement a model in a Stan program;
- Feed in data and generate posterior samples;
- Extract summaries or samples in R.
What to emphasize
- Stan generates the posterior samples, with cmdstanr providing the
interface from R. Subsequent analysis is done in R.
- MCMC is presented in a later episode.
- The basic program blocks answer the following questions:
- data: What do we observe?
- parameters: What do we want to infer?
- model: How are data and parameters related?
- Parameter constraints can be important, and incorrectly specified constraints can prevent the program from working correctly.
- The generated quantities block is where posterior predictions (and other derived quantities) are generated.
- Posterior predictions naturally incorporate both parameter uncertainty and observation-level variability.
- Simulation is an important model-development tool, and failure to reasonably recover the generating parameters can signal issues.
Tips
- Have learners install and test CmdStan before the lesson! Compiler problems are common and can consume substantial teaching time.
- There is no need to spend much time on MCMC mechanics, diagnostics, Jacobians, or code optimization; these can be covered later.
Markov chain Monte Carlo
Instructor Note
The goal of this episode is to provide understanding of how posterior samples are generated, and how the reliability of MCMC sampling can be assessed:
- Understand the basic idea of MCMC;
- Implement a simple Metropolis–Hastings sampler;
- Gain intuition about how the sampling can go awry;
What to emphasize
- MCMC is a means for generating samples from (posterior) distributions.
- The target distribution doesn’t have to be the posterior; the applications are broader!
- In a Markov chain, each sample is generated based on the previous one.
- Code idea: the proposed move is accepted or rejected according to the relative posterior density.
- The proposal distribution affects efficiency.
- Warm-up discards bias of initial part of chain.
- Running multiple chains with distinct starting locations is important to aid mixing.
- Trace plots provide an intuitive visual assessment of mixing; () formalizes comparison of within- and between-chain variation.
- A sampler can appear to behave well but still fail to explore the posterior comprehensively, e.g. for multimodal posteriors.
Tips
- The aim is intuition rather than theory.
- Spend time visualizing chain trajectories. This illustrates concepts such as warm-up, mixing, autocorrelation, and multimodality.
- Online applets such as https://chi-feng.github.io/mcmc-demo/app.html are useful (HMC, NUTS).
- The hand-written Metropolis–Hastings sampler is pedagogical. Learners are not expected to implement their own samplers for real analyses. However, writings samplers is still a relevant skill for many more complex models!
- Connect this episode to Stan: Stan automates the sampling machinery, but users still need to determine sample trustworthiness.
Hierarchical models
Instructor Note
The goal of this episode is to introduce hierarchical modeling as the way to model related groups:
- Represent group-specific parameters as draws from a common population distribution;
- Implement hierarchical models in Stan;
- Understand how the different pooling strategies share information.
What to emphasize
- Hierarchical models are useful when observations naturally belong to related groups.
- Group-specific parameters assumed to arise from a common population distribution.
- This produces partial pooling: estimates for individual groups are informed both by their own data and that from the other groups.
- The amount of data for a group generally affects their estimates proportionally: the less data the bigger the pooling effect towards the population mean.
- The parameters of the population distribution (hyperparameters) are themselves inferred from the data.
- Hierarchical models quantify variation both within and between groups.
- Posterior prediction can be performed both for existing groups and, conceptually, for new groups drawn from the same population.
Tips
- Partial pooling is the central concept. Contrast it explicitly with complete pooling and fitting each group independently.
- It might be a good idea to live code an unpooled model into a partially pooled one by simply by changing the hyperparameters into variables and giving them priors. Such mechanical edits may provide a good starting point for a conceptual discussion: why does this simple change produce partial pooling, and how does the code relate to the corresponding mathematical model specification?