Principal investigator: Eduarda Miller de Figueiredo
Paper title: ARE THE EFFECTS OF INFORMATIONAL INTERVENTIONS DRIVEN BY SALIENCE?
Article authors: Eric Bettinger, Nina Cunha, Guilherme Lichand, Ricardo Madeira
Location of the intervention: São Paulo, Brazil
Sample size287 schools – 19.300 students
Main theme: Education
Main variable of interestSchool attendance and math GPA
Type of interventionSending SMS messages with school information.
Assessment method: Differences in Differences
Evaluation Context
Previous studies demonstrate that information interventions affect fundamental decisions. When provided with specific information, agents begin to make decisions based on these new beliefs, making decisions strictly aligned with optimization. Communication between parents and children is an interesting scenario in this regard, because as children grow, their goals may diverge from those idealized by their parents, making it more difficult for parents to observe their children's efforts in their studies.
In Brazil, it is even more difficult to involve parents in this aspect of their children's lives. The main communication technology between parents and the school is handwritten notes sent by the students themselves, who may not have the right incentives to actually convey the message to their parents. And in most Brazilian states, there are no real-time school information systems for parents to track their children's attendance and behavior.
These information-based interventions have already been shown to be effective in decision-making. Regarding education, Bergman (2017) found that sending SMS messages to parents when their child was not completing assignments resulted in a positive effect on school performance. Similarly, Kraft and Dougherty (2013) showed that frequent phone calls from teachers to parents increase student engagement in the classroom. Given the need for greater parental involvement in their children's school life, an experiment was conducted to send information to parents about attendance, tardiness, and schoolwork. This was considered because receiving a message from the school makes recipients perceive its importance, likely influencing their preferences and beliefs.
Brazil still struggles with the quality of education. State governments are responsible for primary and secondary schools, with the São Paulo state education system encompassing the largest number of students in the country. The entire state has 5,3 million students, of whom 700.000 were in the ninth grade in the year of the study (2019), and of those, 63% were in schools administered by the São Paulo state government.
In 2015, students in the São Paulo public school system scored 4,7 out of 10 on the National Index for the Development of Basic Education (Ideb). According to the authors, this poor performance stems from a combination of precarious infrastructure, low teacher compensation, and low family involvement in students' school life. Even in the sample used for the research, 59% of families earned less than three minimum wages per month, demonstrating the low socioeconomic status of these parents.
Regarding the students who participated in the experiment, 50% are girls, 40% are mixed-race or black, and the average age is 14,7 years. The vast majority of those enrolled in the program are mothers around 40 years old. Of these parents, 69% have only completed high school, and 2/3 of the students in the sample progress in school as much as their parents.
This information demonstrates the need to improve the country's school performance, and to achieve this, it is necessary to understand some factors that can improve the academic performance of these children.
Intervention Details
The intervention will take place through messages to parents. For this purpose, parents from the schools in the sample were randomly selected to receive messages via cell phone with information about their children's school life.
The math teachers at these schools were required to fill out a platform weekly with student information regarding attendance, tardiness, and assignment completion. Within each classroom, parents were randomly selected to receive different text messages (SMS), allowing for the capture of additional effects of the information on parental beliefs and behaviors. These messages will be sent weekly during the second semester of the 2019 school year. The messages sent contained the following content, depending on the group the parents belonged to:
| Boss | Information | Information Regarding |
| For good academic performance, it is important that Guilherme does not miss school without a valid reason. | According to the information recorded by the teacher in the system over the last 3 weeks, Eric missed fewer than 3 classes. | In the last 3 weeks, Nina missed fewer than 3 classes. In his class, most students didn't miss any classes. |
Parents had to provide consent through a registration form, where they provided their cell phone number, relationship to the student, sex, age, race, education, and the child's gender and age. In addition, a voice survey was conducted to collect self-reported parenting practices. At the end of the survey, students were also asked about parental involvement.
Methodology Details
The experiment was conducted in 287 schools in São Paulo, encompassing 19.300 ninth-grade students. Randomization of the groups occurred in two stages. In the first stage, schools were randomly assigned to each of five different groups; in the second stage, students were randomly assigned within the classroom to each type of treatment, as shown in the following figure:
In summary, subsamples A, B, and C allow us to separate the effects of information and salience. Subsample D allows us to estimate the effect of salience without information repercussions, and finally, subsample E allows us to determine the extent of the teacher effects. This last subsample was a requirement of the São Paulo State Department of Education. The pure control group, subsample B, is a randomized counterfactual, and its teachers do not fill out the platform weekly.
A difference-in-differences strategy was used to estimate the effects of the program, and the relevant parameter for this study will be the average treatment effect on the treated individuals.
Result
In the initial research – on parents' parenting practices and beliefs, prior to the intervention – parents generally believed that their children missed fewer classes than they actually did. However, this did not apply to the GPA (obtained through the average of grades), since the sample was equally divided between optimistic, precise, and pessimistic parents.
The Bolsa Família conditional cash transfer program should be considered, as it requires 85% student attendance; however, parents who enrolled in the intervention were less likely to benefit from the program. Even considering this attendance requirement, the program had an effect of 2,1 percentage points, meaning the SMS intervention was equivalent to an increase of five classes in the academic year. Furthermore, the information also impacted the mathematical GPA. An important point to note is that salience messages account for the largest share of the effects, with a difference of 89-126% compared to the information group.
It was also observed that parents who received any of the messages asked their children more questions about school, encouraged their studies more, and had higher aspirations for their children to graduate. Furthermore, the children treated were more involved in academic activities.
An analysis was also conducted to observe whether the effects are short-term, and the results demonstrate that messages containing content about the importance of academic activities do not disappear; in fact, they increase over time. The graph below shows these effects over time:
In summary, several positive effects were found through communication between the school and parents via SMS, modifying parental behavior and leading to greater involvement in their children's school life, which consequently improved their academic performance.
Lessons in Public Policy
The intervention of information about children's school life is yet another hypothesis that behavioral mechanisms can explain why communication with parents works, becoming further evidence suggesting that parents play a crucial role in shaping the behavior and school performance of minors.
Thus, an alternative approach emerges to foster a more enthusiastic relationship between parents and their children's school life, aiming to improve educational outcomes in developing countries.
Reference
BETTINGER, Eric et al. Are the Effects of Informational Interventions Driven by Salience?. 2019.
