Principal investigator: Angelo Cruz do Nascimento Varella
Article title: HOW RESEARCH AFFECTS POLICY: EXPERIMENTAL EVIDENCE FROM 2,150 BRAZILIAN MUNICIPALITIES
Article authors: Jonas Hjort, Diana Moreira, Gautam Rao and Juan Francisco Santini
Location of the intervention: Brazil
Sample size: 2.150 Brazilian municipalities
Main theme: Economic Policy and Governance
Type of Intervention: Experiment with mayors on academic research
Main Variable of Interest: Mayors' decisions
Evaluation method: Experimental Evaluation (RCT)
Policy Problem
An explosion of new studies and research has occurred over the last few decades. According to the American Economic Association (AEA), between 2013 and 2019, there were more than 2.500 registered empirical studies using experiments. This vast application of scientific knowledge has generated increasing and relevant validation of public policies, which have defined positive and negative aspects in various sectors, such as education, income, and health.
However, there is still little knowledge regarding the value given by political leaders and policymakers to the findings of the scientific community. In other words, further progress is needed in mapping and understanding the influence of studies and research on political agents.
Assessment Context
Municipalities are the smallest level of government in Brazil. At the time of the research, there were 5.570 municipalities in 26 states, led by mayors directly elected by their respective populations. Each mayoral position has characteristics similar to other executive branch functions, such as a 4-year term with the possibility of one reelection, appointments of secretaries to handle municipal bureaucracies, and the power to create public policies in areas such as health, education, and transportation. Each municipality has a legislative chamber that, together with the mayor's office, negotiates the budget, which comes mainly from federal transfers (about 60%) and tax revenue (about 15%). Mayors have autonomy to execute the budget and typically face the electoral consequences of accountability to society.
To carry out a large-scale experiment, the researchers partnered with the National Confederation of Municipalities (CNM). This institution, which includes 80% of Brazilian municipalities as members, is a non-partisan organization that acts as a coordinating body and advocates for the interests of Brazilian mayors and municipalities at the state and federal levels. The CNM is responsible for organizing meetings and conferences with various members and entities of the public sector, resulting in the participation of thousands of political agents at its events.
Policy Details
The researchers conducted two experiments with sitting mayors at major CNM events held between 2016 and 2018, with the intention of measuring their willingness to pay for scientific findings, as well as the effect that such discoveries have on the beliefs and decisions of these individuals.
The first experiment involved an analysis of public policy adoption following the dissemination of scientific findings and took place at a large CNM conference in 2016, with the participation of 881 mayors. During the event, participants were informed about research on a public policy that is easy to implement in Brazilian municipalities: sending reminders to taxpayers. The mayors were interviewed 15 to 24 months after participating in the event to verify the implementation of the suggested action.
The second experiment took place over 2017 and 2018, in 14 CNM meetings attended by more than 900 municipal leaders from 657 Brazilian cities. On these occasions, participants were randomly informed of research results on Early Childhood Development (ECD) interventions, which have extensive scientific backing. After the information was disseminated, the researchers conducted a two-stage evaluation, in which the mayors were invited to participate in a lottery system where participants received 100 tickets to compete for a trip to the United States.
Assessment Method
In the first experiment, by informing mayors about the effectiveness of a taxpayer reminder program, the researchers enabled the implementation of a simple and low-cost public policy, mapping the agents who actually chose to adopt the proposed measures through the presentation of favorable scientific findings.
In the second experiment, four empirical studies on DPI (Developmental Intellectual Property) in the United States, Jamaica, Colombia, and Michigan were presented. Researchers questioned mayors about their beliefs regarding the effectiveness of DPI public policies in two of the presented locations and in their respective municipalities. Participants then received 100 tickets to enter a draw for a trip to the United States.
Initially, each individual had the option to use the tickets to, instead of competing in the trip raffle, make an offer to purchase information from scientific studies, as in an auction. The results were revealed to participants whose offer exceeded a minimum value, randomly established for each individual. In the second phase, the mayors received an additional 100 tickets and made a new offer for the results of the three studies that had not been selected in the first round.
At the end of the experiment, the mayors were again questioned about their beliefs regarding the implementation of DPI programs. This allows for a comparison of the mayors' willingness to pay for the results of scientific studies and the influence that new information has on their decision-making processes.
Results
In general, the results indicate that the political leaders surveyed are interested in scientific discoveries, and are willing to change their minds and update their beliefs based on concrete evidence and studies.
In the first experiment, over 38% of the invited mayors chose to participate in the information sessions about the citizen reminder policy. In turn, participation in the event increased the likelihood of implementing the public policy by 10 percentage points, a 33% increase compared to the control group.
In the second experiment, involving travel tickets, mayors and participants were willing to pay an average of 45 tickets (equivalent to $36) to receive the results of scientific studies, with approximately 98% of participants choosing to pay for at least one ticket for the information. It was also found that individuals who paid more for the tickets were more likely to change their opinions based on the information from the acquired studies.
Lessons in Public Policy
In general, the results indicate that presenting scientific findings to political leaders is relevant. In general, when presented with clear information from research and empirical studies, mayors and other political agents at the municipal level shaped their beliefs and opinions to the point of implementing scientific solutions and knowledge. In other words, presenting scientifically based knowledge to political agents is relevant in order to influence the formulation of public policies and the decision-making process of those responsible for their implementation.
Reference
HJORT, Jonas et al. How research affects policy: Experimental evidence from 2,150 Brazilian municipalities. National Bureau of Economic Research, 2019.
