An Update on the Synthetic Control Method as a Tool to Understand State Policy

In 2017, the Tax Policy Center published “The Synthetic Control Method as a Tool to Understand State Policy,” a guide for using the synthetic control method (SCM) as a quantitative adjunct to case studies. With it, analysts can evaluate cases in which there is a single treated unit and no readily available control unit. The method creates a synthetic control by weighting together a small number of control units drawn from a pool of potential donors. Among other things, that small number lets analysts determine if the makeup of the synthetic control is sensible. In this new report, we review the method and revise the guide to account for recent improvements. The guide now includes ways for reducing so-called interpolation bias, a potential problem for the SCM, without inducing extrapolation bias. A very large pool of potential donors can cause overfitting of a synthetic control and biasing estimates of a policy’s effectiveness. While complicated methods can address this, we provide a simple graphical method. On the other hand, a poorly fitted synthetic control can also bias results and we review several methods to correct the problem. We also discuss ways to improve model selection and to extend inference tests beyond a simple null hypothesis of ‘no effect’. Finally, we also provide graphical intuition that connects interpolation bias, extrapolation bias and how they relate to the SCM selecting a small number of control units.