Rough guide to impact evaluation of environmental and development programs

Abstract

Evaluation of programs, either before they are designed or after they are implemented, are increasingly viewed as a critical for learning and improving accountability of public policies. Unfortunately, resource and environmental economists in developing countries have little or no training or guidance on how to conduct such evaluations using sound and rigorous empirical methods. This paper is a “rough guide” for evaluation of programs, projects and policies in the environment and development arena. First, we provide a general overview of the what, how, and why of program evaluation, with particular emphasis on the role of control groups, pre-&-post measurement, and covariate data to define counterfactual scenarios (including formal definition of all terms). Second, we review detailed examples of the four main methods for evaluation – randomized experiments, natural experiments, matching methods, and panel-based DID estimators – with a description of the pros and cons of each method. Third, the guide provides one detailed case study of a SANDEE funded project from South Asia that allows the reader to learn by using data and econometric code to practice and appreciate some of the challenges of impact evaluations. Finally, we conclude by placing the econometric evaluations within the broader context – how can we move beyond estimation of average treatment effects; what do we do under time, resource and data constraints; and when and where should we rely on theory-based evaluations.