Principal investigator: Viviane Pires Ribeiro
Article title: IMPROVING EDUCATIONAL OUTCOMES IN DEVELOPING COUNTRIES: LESSONS FROM RIGOROUS EVALUATIONS
Article authors: Alejandro J. Ganimian and Richard J. Murnane
Location of the intervention: 58 low- and middle-income countries
Sample size: 223 ratings
Sector: Education
Type of Intervention: Evaluation of educational outcomes
Primary variable of interest: Educational outcomes
Evaluation method: Other - Analysis of evaluations of educational interventions
Evaluation Context
Developing countries have become increasingly interested in adopting educational policies that are supported by rigorous evidence of effectiveness. This interest in “evidence-based” policies has resulted in a growing number of impact evaluations since the turn of the century. Consequently, several researchers have attempted to summarize the evidence from these evaluations.
These analyses aim to provide evidence-based support for policymakers, who lack the time (and sometimes specialized training) to examine hundreds of academic articles. The reviews seek to identify promising interventions as well as highlight puzzles to be examined in future research. However, many analyses reach seemingly conflicting conclusions about which educational interventions would improve educational outcomes. Disagreements may arise for various reasons (for example, some authors focus on studies in a specific geographic region, others on studies with cost-effectiveness data, and others on randomized trials). However, even reviews with similar inclusion criteria consider different sets of studies. Furthermore, the different ways in which reviews categorize interventions result in some conflicting conclusions, even when considering the same studies.
Intervention Details
Murnane and Ganimian (2014) analyze and interpret the evidence from 223 rigorous impact evaluations of educational initiatives conducted in 58 low- and middle-income countries. They considered all studies that were recent at the time and that reached seemingly conflicting conclusions about which interventions would improve educational outcomes. To this end, the interventions were grouped based on their theory of action.
The authors reviewed evaluations of educational interventions in preschool, primary, and secondary schools that took place in low- and middle-income countries from 2000 to 2015. The studies included at least one of the following education-related outcomes: enrollment, attendance, repetition, dropout, retention, or measures of student achievement or cognitive skills.
Studies of educational interventions, such as the provision of computer-assisted learning or changes in incentives for teachers, were included, as well as studies of health interventions, such as the provision of medications and iron supplements.
Methodology Details
To conduct the analysis, the authors included only studies that take advantage of exogenous variation in intervention reception, either by randomly assigning individuals to a treatment or by exploiting quasi-random variation in treatment assignment in natural experiments. Studies included based on natural experiments use differences-in-differences, regression discontinuities, and/or instrumental variables to estimate the causal impacts of interventions. Studies that attempt to estimate counterfactual outcomes, either through the use of correspondence methods or fixed effects for individuals in pre- and post-comparison, were not considered.
Results
Murnane and Ganimian (2014) report the effect of interventions on student participation, student performance, or teacher attendance for every US$100. Thus, the authors highlight four lessons from the studies they reviewed:
1. Reducing the cost of attending school and expanding educational options increases attendance and achievement, but does not consistently increase student achievement;
2. Providing information on school quality, developmentally appropriate parenting practices, and the economic returns of schooling affects parental actions and the performance of children and adolescents;
3. More or better resources improve student performance only if they result in changes in children's daily experiences at school;
4. Well-planned incentives increase teacher effort and student performance at very low levels, but teachers with low qualifications need specific guidance to reach minimally acceptable levels of instruction.
Lessons in Public Policy
The study conducted by Murnane and Ganimian (2014) yields some useful lessons for improving educational provision. That is, increasing the quantity or quality of resources, at home or at school, had at best modest effects on student achievement. The same is true for initiatives that have increased the daily school load.
There is evidence that addressing students' learning needs individually, helping teachers personalize instruction, providing additional support to students, or allowing students to learn at their own pace, increases their achievement. Perhaps equally important, these strategies are often more effective in improving the skills of low-achieving students.
Increasing the effort of teachers and/or principals, involving parents and communities in school management, has not been effective in improving student outcomes—especially in environments where pre-existing capacity is low. Offering incentives to principals and/or teachers has improved school performance. However, it has proven difficult to design incentive systems that do not produce dysfunctional responses. Hiring teachers with renewable fixed-term contracts has increased student performance in the short term, but scaling this policy and maintaining it over time presents many challenges.
Professional development aimed at increasing teachers' knowledge of content and pedagogy did not improve student performance. Pre-prepared lessons for teachers improved student performance in low-capacity teacher environments.
The consequences of any school improvement strategy will likely depend on the specific environmental circumstances. In settings where many children are out of school, it may make sense to focus on demand-side interventions. However, in countries where most children attend school and learning is low, improving instruction is crucial. Similarly, in settings where teacher ability is very low, making their job easier—for example, by tracking students by ability or preparing lessons for teachers—may produce incremental improvements. In countries where teachers have moderate ability but low effort, aligning the incentives of students, parents, and teachers may make more sense, provided there are mechanisms to discourage, identify, and correct dysfunctional responses.
Much of the information about the interventions concerns short-term outcomes for students. Examining the long-term consequences of promising interventions in developing countries is an important goal for future research.
References
MURNANE, Richard J.; GANIMIAN, Alejandro. Improving educational outcomes in developing countries: Lessons from rigorous impact evaluations. NBER Working Paper, no. w20284, 2014.
