Principal investigator: Bruno Benevit
Authors: Mounir Karadja and Erik Prawitz
Original title: Outsourcing Education: Experimental Evidence from Liberia
Location of the Intervention: Liba © ria
Sample Size: 3,508 students, 185 schools
Sector: Economics of Education
Primary Variable of Interest: Quality of schools
Type of Intervention: private management
Methodology: OLS
Summary
Public-private partnerships (PPPs) can present opportunities for public managers to improve the delivery of public services to the population, taking advantage of incentives for efficiency in the private sector. The program Partnership Schools for Liberia (PSL) was a case study of PPP in the education sector, randomly delegating the management of 93 public schools to private organizations. The objective of this study was to evaluate the impact of this program on the management, teaching performance, and student satisfaction of the selected schools. The results indicated that students from these schools performed better on English and mathematics tests and that students and families showed greater satisfaction in PSL schools. Although significant heterogeneity was identified among the providers, no heterogeneities were identified in learning gains or enrollment by student characteristics.
- Policy Problem
Public-private partnerships (PPPs) provide opportunities for public managers to improve the delivery of public services to the population, allowing them to delegate the management of the supply of goods and services to the private sector. However, while PPPs have the potential to induce greater efficiency due to the incentives faced by the private sector, there are reservations regarding how private institutions can achieve such efficiency through cost cuts, potentially penalizing the quality of the contracted services (ROMERO; SANDEFUR; SANDHOLTZ, 2020).
In Liberia, program Partnership Schools for Liberia (PSL) was a case of PPP in the education sector implemented in the country in 2016, where the management of 93 public schools was randomly delegated to eight private organizations. In this sense, this program provided an experiment for evaluating the performance and cost-benefit of PPPs in the education sector, making it possible to observe how the potential conflict between the quality and efficiency of schools is dealt with through private school management.
- Policy Implementation Context
The study conducted an experiment to designate the schools participating in the program. The PSL schools, the treatment group, were managed by private organizations, while the control group schools remained publicly run. As stipulated in the contract, the PSL initially guaranteed one teacher per grade in each school, in addition to extra funding, and had authorization to limit class sizes. Finally, while both the program schools and traditional public schools remained free for primary education students, the public schools charged early childhood education fees (ECE).
Random sampling and assignment were conducted for a study analyzing public primary schools in Liberia. It was established that schools to be part of the PSL program needed to meet certain criteria, including a minimum number of classrooms and teachers, adequate road access, and single-shift operation. After the selection process, only a few schools met all the criteria. The schools participating in the study, on average, were closer to the capital, had more resources, and better infrastructure than the average school in the country. Schools were paired within each district based on available resources. Some schools initially designated as part of the program were not managed by private providers and were therefore considered non-compliant.
- Evaluation Details
Random sampling and assignment established that schools to be part of the PSL program needed to meet certain criteria, such as a minimum number of classrooms and teachers, adequate road access, and single-shift operation. After the selection process, only a few schools met all the criteria. The schools participating in the study, on average, were closer to the capital, had more resources, and better infrastructure than the average school in the country. Schools were paired within each district based on available resources. Some schools initially designated as part of the program were not managed by private providers and were therefore considered... noncompliers to the treatment, allowing the identification of the intention-to-treat (ITT) effect from observations of these schools (7 schools in total).
Data collection in the participating schools took place at the beginning of the school year, September to October 2016, and at the end of the school year, May to June 2017. The data collected included information regarding invariant characteristics of the students and their performance on English language and mathematics tests, as well as the characteristics of the schools and the teaching strategies of the English language and mathematics teachers. The data also showed differentiation in terms of the format of the questions tested: standardized, conceptual, and abstract. Additionally, to verify the student composition among the schools, the research compiled enrollment data from 20 students per school in the school year prior to the experiment.
- Method
To identify the impact of the PSL program on student performance, the study employed two ordinary least squares (OLS) models, estimating the treatment effect on those treated (ToT) and the intention-to-be-treated effect (ITT). To estimate ToT, the identification of schools initially assigned as treated was used as an instrument. The first model consisted of a simple comparison of post-treatment outcomes for students in both groups. The second specification adds controls for time-invariant characteristics measured at the student and school levels. Both specifications are estimated by pooling the standard errors at the school level.
Subsequently, the impact of the PSL program on access to education was examined. Impacts on three outcome variables were observed, considering data from the school year prior to the experiment: enrollment, student attendance, and selection of students with disabilities. Models were estimated at the school and student levels. Impacts on these outcome variables were also estimated considering class sizes restricted due to PSL provider requirements, as well as the impact on both restricted and unrestricted class sizes on knowledge test performance. Again, standard errors were pooled at the school level.
The study also presents ITT analyses on school management, teacher behavior, student family behavior, fee collection, and student attitudes. Finally, the study presents a heterogeneity analysis among the eight private providers. Thus, the impact between schools of each provider was observed (i) on performance in knowledge tests, (ii) on school behavior in teacher retention and hiring, and (iii) on parent and student satisfaction.
- Main results
The results indicated that after 1–2 months of treatment, students' test scores increased significantly by 0,05 standard deviations in mathematics and 0,07 standard deviations in English. The treatment effect increased over time, indicating continuous gains. The treatment effects were significant for new modules and conceptual questions, suggesting real learning gains. No evidence of heterogeneous treatment effects by socioeconomic status, gender, or grade level was found. Additionally, the results were robust to different measures of student ability.
Changes in enrollment between treatment and control schools were observed. Before treatment, treatment schools were slightly larger, with an average of 34 more students. During the 2016-2017 academic year, treatment schools had, on average, 57 more students than control schools. This resulted in a net increase of 25 students per school. Student attendance also improved in treatment schools, with a difference of 16 percentage points (pp) compared to control schools. There was no evidence of systematic exclusion of student groups in treatment schools. Analysis of student selection showed no statistically significant differences between treatment and control schools. When class size restrictions were considered, the program led to a significant decrease in enrollment. According to the authors, this behavior did not appear to be motivated by the selection of “best” students, but rather by the limitation of class sizes by providers and the elimination of double shifts.
Regarding school management, it was found that students in PSL schools were 23 percentage points more likely to have a textbook and 8,2 percentage points more likely to have writing materials (pen and notebook). PSL schools were 8,7 percentage points more likely to be open during the school day and had longer school days, with an additional 3,2 hours per week. Although principals in PSL schools had similar scores to those in traditional public schools, they dedicated more time to management activities. Management practices in PSL schools were significantly better, suggesting a shift in the role of principals in these schools.
Teacher behavior was analyzed to assess accountability and the effects of school management. Teachers in PSL schools were more likely to be present during surprise checks (20 pp) and less likely to be absent in the preceding weeks (7,5 pp). Furthermore, student reports indicated that teachers in PSL schools were more likely to be actively involved in instruction (15 pp), less likely to stray from the subject than teachers in PSL schools (25 pp), and less likely to assault students (6,6 pp).
Results regarding student and family approval show that the program also increases the proportion of satisfied parents (7,49 pp) and the likelihood of students finding school enjoyable (5 pp). Although PSL schools cannot charge fees, there are still reports of parents paying some fees. Family expenses on education decrease, mainly due to the free provision of textbooks and uniforms by providers. However, there is no change in parental involvement in their children's education. Regarding student attitudes, those in PSL schools are more likely to see school as useful and less likely to have negative opinions about some tribes in Liberia.
Regarding the effects according to each of the eight providers, the PSL schools showed heterogeneity in terms of the magnitude and significance of their impact. The treatment effects on composite test scores are positive and significantly different from zero for three providers, and effects are positive and statistically insignificant for another three providers. The schools of the other two providers showed negative and statistically insignificant results. While there was an increase in the number of teachers assigned to the schools, there was also an additional exodus of teachers. According to the authors, although dismissing ineffective teachers is important, a redistribution of teachers is an unlikely policy to improve the average performance of the Liberian education system as a whole.
- Lessons in Public Policy
This article investigated the effects of a Public-Private Partnership (PPP) on education in Liberia, focusing on improving the quality of and access to education. The PSL program involved the private management of public schools, with the expectation of improving student performance and increasing parental satisfaction. To this end, classroom observations, teacher interviews, and data analysis of students and families were conducted to assess the program's effects on teacher behavior, student and parental satisfaction, as well as enrollment and academic performance.
The results showed that, although there were significant improvements in teacher behavior and student and parent satisfaction in some schools, the program's effects were heterogeneous among providers. Schools managed by some private providers experienced significant increases in test scores, while others had insignificant results. Furthermore, the program reduced school fees, teacher selection and retention, as well as student and family enrollment and satisfaction.
These results highlight the importance of considering heterogeneities and incentives of private management when formulating educational policies based on public-private partnerships. Understanding these differences is fundamental for developing effective strategies that can improve the quality of and access to education, especially in resource-limited contexts of developing countries like Liberia.
References
ROMERO, M.; SANDEFUR, J.; SANDHOLTZ, WA Outsourcing Education: Experimental Evidence from Liberia. American Economic Review, v. 110, no. 2, p. 364–400, 1 Feb. 2020.
