Principal investigator: Eduarda Miller de Figueiredo
Location of the Intervention: Jerusalem
Sample Size: 880 students
Sector: Education
Primary Variable of Interest: Test score
Type of Intervention: Budget increases for education
Methodology: Differences in Differences
Summary
The question of how teacher characteristics affect student learning has long been a concern for economists, educators, and parents. The literature suggests that parents would be willing to pay for more qualified teachers, thus highlighting the need to estimate the effect of teacher characteristics on student grades. To further understand this relationship, the authors conducted a Difference-in-Differences analysis using Jerusalem schools where teachers received a training program. The results suggest that schools in the treatment group obtained higher scores on applied mathematics and literature tests.
- Policy Problem
A study by Black (1999) suggests that parents would be willing to pay for more qualified teachers if they knew it would lead to their children learning more. From this, numerous studies have emerged attempting to estimate the effect of teacher characteristics on student grades. Another line of research observes how specific on-the-job training affects student performance, where on-the-job training may be no less important than the more widely studied measures of general education or experience. According to Farrell and Oliveira (1993), initial training is essential for teaching the subject matter, while on-the-job training is essential for teaching teaching skills.
The aim of this article, analyzed here, was to contribute to the literature on the effects of in-service training through an evaluation of teacher training in elementary schools in Jerusalem. To this end, the causal effect of a training program designed to improve the teaching of language and math skills on the results of tests administered to students was estimated.
- Implementation and Evaluation Context
Public schools in Jerusalem and elsewhere in Israel are separated into religious and secular systems. In 1995, some Jerusalem public schools received a special amount of funding primarily intended for on-the-job teacher training. This made it possible to provide a type of training to reading and mathematics teachers based on widely used pedagogical strategies originally developed in United States schools – “humanistic mathematics” and “individualized instruction.”[1].
Schools were selected for two neighborhoods in North Jerusalem.[2]In addition to other towns and cities in Israel, these two selected neighborhoods form a distinct geographic block, constituting an autonomous school district with a high proportion of immigrant students and performance levels below the city average.
- Policy/Program Details
The intervention aimed to increase pass rates, improve performance test scores, and enhance the school climate. The intervention took the form of budget increases, although in some cities the money was earmarked for specific uses. In Jerusalem, the money was primarily spent on teacher training, which was conducted jointly with all teachers of relevant subjects in a given grade level, and all participated. The authors report that they confirmed that these teachers were the same ones who taught the children, with the exception of one who retired.
In Jerusalem, a total of 10 primary schools were affected, 7 secular and 3 religious.[3]The authors present data for three dates: 1994, which precedes the start of the training program; 1995, the year the training began; and 1996, a year in which training inputs remained at a new, higher level in the schools studied.
Training is measured as the total hours of instruction received by all teachers in each school per week, on average throughout the school year, and is recorded separately for reading and mathematics. The data showed a sharp increase in training hours offered to teachers in secular treatment schools between 1994-1995 and 1995-1996. The authors point out that, in contrast to the data on training and expenditure, instruction hours per student and class size remained essentially unchanged in both treatment and control schools.
The tests were based on the core material taught throughout the year in each grade. However, although the intervention took place school-wide, the only students for whom data on test scores before and after the start of the intervention are available are the cohort of fourth-grade students enrolled in the 1993-1994 school year. Therefore, the study is limited to fourth-grade students from 1994 in the schools.
The control schools were selected for comparability and practical reasons related to data collection. Of the 20 schools that applied to be included in the control group, 11 met the criteria for inclusion in the study.
- Method
The study was conducted for the two types of schools separately – religious systems and secular systems. Thus, 406 students took the reading test and 428 took the math test in the secular treatment schools, while in the secular control schools, 405 students took the reading test and 420 took the math test. Therefore, the evaluation strategy compares the population of fourth-grade students enrolled in treated schools in 1994 with the population of students in the control schools.
The authors used a Differences-in-Differences model to perform the impact analysis. Thus, they rely on the assumption that any differences between students in the treatment and control groups are fixed over time; therefore, repeated observations of the same students can be used to make the treatment and control groups comparable. The main variable of interest is the student's test score.
The comparisons of means by treatment status performed in the study show that students in the treatment schools had lower test scores than students in the control schools in 1994, before the start of treatment. Thus, students in the two groups are not directly comparable. To correct for this bias, the fixed effects were replaced by the assumption that students in the treatment and control schools are conditionally comparable based on prior test scores and other observed covariates.
- Main results
The results for secular schools show no significant change in test scores from control schools between 1994 and 1996. In contrast, test scores increased considerably and significantly in treatment schools. Thus, the estimate is for an increase of 0,62 in reading test scores and 0,46 in math test scores for the period from 1994 to 1996.
Estimates for religious schools show a sharp decline in treated schools. However, treatment in these schools began later and the scale of intervention was smaller; therefore, the authors emphasize that their results appear to be less reliable and more difficult to interpret. Results controlling for the bias explained earlier demonstrate that most of the difference in 1994 scores between treatment and control students remains even after applying the bias control.
When a subsample of math test scores was used, the findings were similar to those of the full sample, thus corroborating the results found previously.
- Lessons in Public Policy
The results found by this study suggest that teacher training may provide a less costly means of increasing test scores among public school students, highlighting that such intervention could be an alternative to reducing class sizes or increasing school hours.
References
ANGRIST, Joshua D.; LAVY, Victor. Does teacher training affect pupil learning? Evidence from matched comparisons in Jerusalem public schools. Journal of labor economics, v. 19, no. 2, p. 343-369, 2001.
[1] Translated from: “[…] 'humanistic mathematics' and 'individualized instruction'”.
[2] Neve Yaakov and Pisgat Zeev.
[3] The authors point out that one of the religious schools was excluded from the analysis because it was opened after the intervention began.
