Bayesian Causal Inference
Preface
There is a reputed developer somewhere in Nebraska who maintains a pivotal portion of the contemporary open-source ecosystem. The whole structure depends on their work. A Jenga tower of software components harnessed together and stacked atop their infrastructure. Whether the legendary figure exists is somewhat immaterial, the point is that the architecture of modern software and scientific computing depends on the distributed work of innumerable contributors, latched together by a glue of shared conventions and protocols. This book similarly owes a debt to the community of statisticians and practising scientists who have built and battle-tested contemporary probabilistic programming languages. Libraries, such as PyMC, Stan, NumPyro, Turing, provide a lexicon for generative modelling. The book is a love letter to the community that built these tools, and an attempt to distill what we have learned from them.
It is only apt then that the subject of this book is fundamentally about the compositional building blocks of structured learning. Bayesian causal inference can be intimidating because it inherits two layers of abstraction: the mathematical machinery of probability theory, and the conceptual machinery of causal inference. The goal of this book is to demystify both layers by breaking them down into their atomic components, and showing how they provide a rich expressive language for the ground-up construction and evaluation of scientific theories.
The Figure 1 maps the vocabulary of both probability distributions and causal structure. The left panel shows some of the distributional atoms from which the models in this book are assembled. The right panel highlights the structural patterns — fork, chain, and collider — that constrain what those models can say.
Causal questions ask what would happen if we intervened: if a patient received a different dose, if a policy were enacted, if a voter deliberated independently. Answering such causal queries requires more than fitting a model to data. It requires that we: (1) specify which variables cause which and (2) identify the assumptions that warrant a causal reading of the relationships, while choosing an estimation strategy that respects both.
The contemporary understanding of the patterns and edge cases of defensible causal inference is the product of researchers across a range of disciplines — Pearl’s do-calculus from computer science, Rubin’s potential-outcomes framework from statistics, the econometric tradition’s attention to endogeneity and identification, epidemiology’s focus on confounding and selection. No single tradition owns the full picture. This book borrows from all of them, magpie-like, taking what is useful and attempting to align them within a single Bayesian lens. Fortunately, these views are complementary and all well-phrased in the idiom of probabilistic programming.
This book draws on the modelling strategies that have emerged from these traditions of structural learning: propensity scores, instrumental variables, structural equation models and discrete choice models for cross-sectional data; pre-post designs, interrupted time series, synthetic control, fixed effects and marketing mix models for panel data. We show how each can be expressed, fitted, and interrogated within a unified Bayesian framework. These methods are not interchangeable tools in a kit. With cross-sectional data they form an escalating sequence: each one addresses a harder threat to causal validity than the last, and each demands a richer joint distribution to do so. With panel data the threat holds still, but we can leverage more data and opportunities to clarify the evidence base of the counterfactual claims.
As the models grow more expressive, the assumptions required to identify causal effects grow stronger, and more contestable. The DAG — initially a convenient sketch of which variables affect which — becomes load-bearing architecture. Simpler models tend to hide their commitments in the machinery. This trade-off between expressive power and assumption strength is a recurring theme of the book, and learning to navigate it honestly is much of what causal inference is about. Seen this way, probability distributions and DAG patterns are the alphabet of a shared language, and the compositional rules for assembling them are the grammar. A probabilistic model is an argument about the state and structure of the world. This book is an introduction to the language which expresses those structures, and a guide to using it for statistical inference and scientific discovery.
Book Structure: Topics covered
The paths through the book can be seen in Figure 2.
There is a narrative through-line developed across the book. The chapters are ordered to build on one another in an escalating sequence. The foundations establish the Bayesian workflow and the causal language it operates on. The cross-sectional methods climb from selection and observed confounding, through unobserved confounding and a misspecified graph, to the adequacy of counterfactual projection. They close on sensitivity analysis, which measures how much weight an unverifiable assumption bears. The panel data methods also span different threats to the validity of counterfactual projection. We must probe whether a unit’s own history extrapolates, whether the comparison units are the right ones, and whether the model asks more of the data than the data can give. The model structures make increasingly ambitious use of the available information, ending where observational data alone cannot carry the claim and an experiment has to. The standard set by sensitivity analysis carries across both parts. Working through the book, you will learn a series of causal identification strategies and implement them in a probabilistic programming language. Additionally, you’ll be able to articulate what those strategies assume, stress-test those assumptions, and report the credibility of the findings with compelling arguments and robust statistical evidence.
This book is for applied researchers, data scientists, and statisticians who fit models and want to know when those models can support their causal claims. It offers a unifying Bayesian lens on the core tasks of causal inference, built on the conviction that the analyst’s job is not to defend assumptions but to make them explicit and measure what they imply. It assumes you have some familiarity with Bayesian inference, but signals the prerequisites for each chapter so you can navigate the book in a way that suits your background and interests.
Software and Code
Throughout this book we will illustrate the statistical modelling and causal inference techniques using the probabilistic programming language PyMC1 and (where appropriate) packages from the PyMC-ecosystem such as CausalPy2 and PyMC-Marketing. These packages are all built on top of the PyMC PPL. Our preference will be to explicitly express the models with PyMC for the pedagogical clarity it offers, but if the context is clear and one of the packages provides a convenient wrapper around the intended model, we may adopt the package for demonstration. Throughout we will ensure that all the code in this book is explained and self-documenting, but it’s worth noting that all the packages in the PyMC-ecosystem come with extensive supporting documentation and can serve as useful references for deeper-dives. The software is built and maintained by a vibrant community of researchers and practising scientists who are always happy to receive PRs on github. If you are reading the hard copy book we may have hidden some boiler plate code for sake of readability, but the full code will be available online at the hosted version of the book.
Acknowledgements
We wish to state our sincere appreciation to the many peers and colleagues who generously shared their time, expertise, and thoughtful feedback during the development of this book. Their careful reading of early drafts, insightful comments, and constructive suggestions have significantly improved the clarity, accuracy, and presentation of the material. In particular we want to thank (for their time and tolerance): Jesse Grabowski, Chris Fonnesbeck, Juan Orduz, Erik Ringen, Carlos Trujillo, Joe Wilkinson, Kelsey Barton Henry, Timothy Mcwilliams, Thomas Pinder, Daniel Lee, Mengxing Baldour-wang, Colt Allen, Bill Engels, Jean van Dyk, Allen Kennedy, Aadil Kurji, Fenner Tanswell, Jessica Olsen. We gratefully acknowledge the collective work of PyMC’s developers and contributors, whose efforts helped make this research possible. Any errors or omissions that remain are entirely our own.