Impact evaluation in international development : theory, methods, and practice / Paul Glewwe and Petra Todd.

Format:

Book

Author/Creator:

Glewwe, Paul, 1958- author.

Todd, Petra, author.

Language:

English

Subjects (All):

Economic development projects.

Physical Description:

1 online resource (425 pages)

Edition:

First edition.

Place of Publication:

Washington, DC : The World Bank, [2022]

Summary:

Impact evaluations are studies that attempt to measure the causal impact of a project, program, or policy on one or more outcomes.This book provides a comprehensive exposition of how to conduct impact evaluations.

Contents:

Front Cover

Contents

Preface

About the Authors and Contributors

Abbreviations

Part I The Basics of Impact Evaluation

Chapter 1 The Purpose of Impact Evaluation

Introduction

The difference between monitoring and evaluation

A definition of impact evaluation

Brief overview of types of impact evaluations

Other issues regarding impact evaluation

Conclusion

Notes

References

Chapter 2 How to Conduct an Impact Evaluation: Getting Started

Step 1. Defining the program and the outcomes of interest

Step 2. Forming a theory of change to refine the evaluation questions

Step 3. Depicting a theory of change in a results chain (logic model)

Step 4. Formulating specific hypotheses for the impact evaluation

Step 5. Selecting performance indicators for monitoring and evaluation

Chapter 3 The Evaluation Problem

Correlation does not imply causation

Potential outcomes and the evaluation problem

Observed outcomes and the gain from treatment

Parameters of interest

Chapter 4 Validity: Internal, External, and Trade-Offs

Internal validity

External validity

Trade-offs and intermediate approaches

Chapter 5 Overview of Impact Evaluation Methods

Using randomized controlled trials to evaluate program impacts

Impact evaluations based on nonrandomized and quasi-experimental data

Part II Experimental Methods

Chapter 6 Introduction to Randomized Controlled Trials

The basic idea of a randomized controlled trial

How does randomization solve the evaluation problem?

What if some people assigned to the treatment group choose not to participate?.

Intention-to-treat effects

Intention-to-treat effects when effects spill over onto nonparticipants

Encouragement designs

Chapter 7 Regression Methods for Randomized Controlled Trials

Estimating average treatment effects when no problems occur

Estimation when some in the treatment group are not treated

Complications caused by sample attrition

Methods for increasing precision of the estimates

Methods for obtaining correct standard errors

Other useful advice and recommendations

Chapter 8 Practical Advice for Implementing Randomized Evaluations

Potential problems with randomized experiments, and possible solutions

Practical advice for randomizing into treatment and control groups

The use of pre-analysis plans in impact evaluations

Other practical advice

Increasing external validity

Chapter 9 Sample Size, Sample Design, and Statistical Power

Statistical power as a criterion for choosing the sample design

Power and MDE calculations in more complex settings

Practical issues regarding power calculations

Further statistical issues

Chapter 10 Recommendations for Conducting Ethical Impact Evaluations

Two frameworks for conducting ethical evaluations and research

Confidentiality

Ethics of randomized controlled trials

Conflicts of interest

Ethical research in practice

Part III Nonexperimental Methods

Chapter 11 Regression Methods for Nonrandomized Data: Cross-Sectional and Before-After Estimators

Examples: Cross-sectional, before-after, and difference-in-differences estimators

Parameters of interest.

The cross-sectional estimator and sources of bias

The before-after estimator

Chapter 12 Regression Methods for Nonrandomized Data: The Difference-in-Differences Estimator and the Within Estimator

The difference-in-differences estimator

Within estimators

Applications of difference-in-differences and within estimators

Chapter 13 Matching Methods

Two simple examples

Cross-sectional matching

Implementation of propensity score matching estimators

Difference-in-differences matching

Additional topics for matching methods

Empirical applications of matching estimators

Chapter 14 Regression Discontinuity Methods

Intuition for regression discontinuity methods

Identification of treatment effects under "sharp" and "fuzzy" data

Checking the validity of a regression discontinuity design

The Hahn, Todd, and van der Klaauw estimation method

Examples of regression discontinuity methods

Chapter 15 Instrumental Variables Estimation and Local Average Treatment Effects

Two uses of instrumental variables estimation for impact evaluation analysis

Instrumental variables estimation of ATE and ATT

Using IV methods to estimate local average treatment effects

Chapter 16 Control Function Methods

The basic idea of the control function approach

Methods for estimating control functions

Standard error calculations for control function estimation methods

Comparing control functions to matching methods and instrumental variables

Adapting the control function approach for estimating ATE(X) and ATE.

An application: The performance of public and private schools in Chile

Chapter 17 Quantile Treatment Effects

The basic idea of quantile regression, with an example

Conditional and unconditional quantile treatment effect estimators

Conditional quantile treatment effect estimators

Unconditional quantile treatment effect estimators

Standard errors

Examples of applications

Part IV Data Collection and Project Management

Chapter 18 Designing Questionnaires and Other Data Collection Instruments

General principles and recommendations

General advice on the design of questionnaires

Household questionnaires

Service provider questionnaires

Community (and price) questionnaires

Other data collection instruments

Paper questionnaires versus computer-assisted personal interviewing

Chapter 19 Data Collection and Data Management

The steps involved in data collection and data management

Establish procedures for collecting and managing the data

Collect the data (including monitoring of data quality)

Further checks of data quality after the fieldwork

Create data files for analysis and dissemination

Establish a system to store, revise, and disseminate the data

Chapter 20 Survey Management

Budgeting and developing an overall plan of activities

Human resources (personnel) management

Logistical coordination

Community relations

Lessons from unfortunate experiences

Note

Part V Related Topics

Chapter 21 Dissemination of Results and Working with Policy Makers

What products should the impact evaluation deliver?.

Dissemination of the findings

Working with policy makers

Chapter 22 Qualitative Approaches, Data, and Analysis in Impact Evaluations

Joan DeJaeghere

Contributions and challenges in using qualitative research in impact evaluations

Different purposes and types of qualitative approaches

The most common methods for collecting qualitative data

Exploratory and explanatory qualitative approaches

Practical suggestions for designing, gathering, and analyzing qualitative data

Annex 22A Questions for realist evaluations

Chapter 23 Cost-Benefit Analysis and Cost-Effectiveness Analysis

Calculation of costs

A simple comparison of cost-benefit analysis and cost-effectiveness analysis

Cost-benefit analysis (valuing the benefits)

Cost-effectiveness analysis

Boxes

Box 1.1 Key organizations and agency departments that focus on impact evaluation in international development

Box 3.1 Requirements for answering the evaluation problem, "What is the causal impact of the program (or project or policy) on the outcomes of interest?"

Box 22.1 Case studies and comparative qualitative analysis: Example of Akazi Kanoze Youth Livelihoods Project (Alcid 2014)

Box 22.2 Longitudinal and theory-based design and analysis: Example of Learn, Earn, and Save Initiative of Youth Livelihoods Programs in Tanzania and Uganda

Figures

Figure 2.1 A results chain diagram: Basic layout and components

Figure 2.2 A more detailed view of what goes into a results chain (logic model)

Figure 2.3 Examples of a results chain's components: Education and health sectors

Figure 2.4 Example of a results chain: Mexico's PROGRESA program.

Figure 2.5 Example of a detailed, successive results chain: Regional vaccination program.

Notes:

Includes bibliographical references.

Description based on publisher supplied metadata and other sources.

Description based on print version record.

Other Format:

Print version: Glewwe, Paul Impact Evaluation in International Development

ISBN:

9781464814983

1464814988

OCLC:

1311330451

Publisher Number:

10.1596/978-1-4648-1497-6