Principal investigator: Angelo Cruz do Nascimento Varella
Article title: ARE THE EFFECTS OF INFORMATIONAL INTERVENTIONS DRIVEN BY SALIENCE?
Article authors: Eric Bettinger, Ricardo Madeira and Guilherme Lichand
Location of the intervention: São Paulo, Brazil
Sample size: 19.253 ninth-grade students from 287 schools in the State of São Paulo.
Sector: Education
Type of Intervention: Large-scale experiment with information systems via text messaging.
Main Variable of Interest: School performance and attendance
Evaluation method: Experimental Evaluation (RCT)
Policy Problem
Brazilian education has made substantial progress in the last two decades, particularly regarding universal access to basic education. However, in terms of the quality of teaching, there is still much room for improvement. Unsatisfactory results can be explained by factors such as poor infrastructure, low value placed on teachers, and limited family involvement in school life. These aspects have a direct impact on student performance and grades, as well as on class attendance and participation, which reflects the quality of education offered and absorbed.
Specifically regarding family involvement in school life, research indicates that increased family interest in students' daily lives has a positive impact on academic performance, as well as classroom attendance. Professionals and scholars affirm that parental involvement is one of the main causes of successful education. Although the best way to encourage this engagement, or the reason for this phenomenon, is still uncertain, it is estimated that effective communication between educational institutions and families is an excellent way to foster this beneficial behavior.
Assessment Context
In contrast to the encouraging success of Brazil's strategy for universal access to basic education, which in 2015 saw more than 98% of young people aged 7 to 14 regularly enrolled in school, the performance of students in the Programme for International Student Assessment (PISA) is worrying. This is because, in the same year, 15-year-old students who participated in the test obtained an average score in mathematics 121 points lower than the average of countries in the Organisation for Economic Co-operation and Development (OECD), indicating a two-year lag in skills related to the subject. Furthermore, 48% of Brazilian students had missed at least one day of school in the week prior to the test, compared to an average of 20% of students from OECD countries.
This performance can be explained by the low level of family involvement in early childhood education. According to the National School Health Survey (PeNSE) by the Brazilian Institute of Geography and Statistics (IBGE), about 25% of relatives do not know if their dependents are absent from school, and only half are informed about their schoolwork and homework. Furthermore, few Brazilian schools have real-time information systems about their students, so the lack of communication between these institutions and families is widespread.
In São Paulo, where the experiment took place, social inequality represents a significant challenge. Besides being the richest and most populous Brazilian state, São Paulo also has the largest national education network, with over 5,3 million students in primary and secondary education in 2015, according to the Ministry of Education's School Census. While wealthier families prefer high-quality private educational institutions, students from lower socioeconomic classes predominantly use public education. In the research sample, more than half of the households had a monthly family income of up to three minimum wages, and about 60% of the heads of household had not completed secondary education.
Policy Details
In partnership with the São Paulo State Department of Education, researchers conducted a large-scale experiment to measure the effect of different communication strategies with the families and guardians of ninth-grade students on their school performance and attendance. The aim is to investigate whether different information mechanisms are capable of influencing the beliefs, aspirations, and behaviors of family members and students.
To do this, the researchers divided the students into five groups, so that their respective families received weekly text messages with different types of information:
- Individuals – specific information about the attendance, lateness, and completion of tasks by their dependents;
- Relative information – specific details about the attendance, tardiness, and completion of assignments of your dependents in relation to the other students in the class;
- Emphasis – general information about the importance of classroom attendance and commitment to education;
- Engagement – specific information about extracurricular activities related to your dependents;
- Control Group – did not receive any information.
The experiment lasted 6 months, from June to December 2016, and included 287 schools, 934 classes, and 19.253 students. The information was provided by the students' mathematics teachers via weekly text messages, and the frequency of the messages varied according to the research needs. The results were supplemented by questionnaires.
Methodology Details
Each group in the experiment has distinct objectives, so the researchers conducted several tests throughout the process. The purpose of the individual information is to ascertain whether knowledge about the dependent's performance is sufficient to interest the family member, altering their behavior. Similarly, the relative information aims to ascertain whether the effect of comparison with the student's classmates is relevant to family involvement.
Engagement data tests the ability of messages to alter family behavior based on stimuli independent of teacher action. Salience data aims to measure the effect of generic, low-cost messages in capturing family attention and generating beneficial behavioral changes among caregivers, efficiently. Control groups serve as a basis for comparison.
Results
Based on the research conducted, it was possible to ascertain that weekly communication with families, through text messages, had a significant impact on student performance and attendance. Groups that received general information about the importance of student monitoring (salience) obtained the best results observed.
Regarding frequency, groups that received individual, relative, and salience information experienced a 2,1 percentage point increase, representing five additional days of classes per year. Average math test scores for students in these groups also increased by 0,1 standard deviation, as did their chances of passing the school year, which increased by 3,2 percentage points. Analyzing only the groups that received weekly information, those that were provided only with generic (salience) information obtained considerably higher results, ranging from 89% to 126% more than those with specific and relative information.
Lessons in Public Policy
Constant communication between educational institutions and families is an excellent way to engage parents in their children's daily lives, improving their academic performance and attendance. Actions that encourage the interest and attention of these individuals in the daily lives of students have the potential to generate benefits with little investment and minimal costs, representing an effective and easily implemented strategy.
Family beliefs and expectations regarding students prove to be useful tools in the pursuit of improving national education, leading relatives and guardians to participate in and improve the study conditions of young people, which represents a significant opportunity for public education policies.
Reference
BETTINGER, Eric et al. Are the effects of informational interventions driven by salience?. University of Zurich, Department of Economics, Working Paper, no. 350, 2021.
