Principal investigator: Viviane Pires Ribeiro
Paper Title: Building Resilient Health Systems: Experimental Evidence from Sierra Leone and the 2014 Ebola Outbreak
Authors: Darin Christensen, Oeindrila Dube, Johannes Haushofer, Bilal Siddiqi and Maarten Voors
Location of the Intervention: Serra Leoa
Sample Size: 5.080 households
Main theme: Health Insurance
Main Variable of Interest: Health system
Type of InterventionCommunity monitoring and non-financial rewards
Methodology: Randomization
Can improving the perceived quality of healthcare promote community health and ultimately help contain epidemics? Seeking to answer this question, Christensen et al. (2021) used a field experiment in the context of the 2014 Ebola crisis in West Africa to evaluate two programs aimed at improving the utilization of government-run clinics and the quality of care provided in these facilities. One program focused on community monitoring, and the other program awarded non-financial prizes to clinic staff. The results generally indicate that promoting accountability not only has the power to improve health systems in normal times but can also make them more resilient to emerging crises.
Evaluation Context
Developing countries are characterized by high mortality and morbidity rates. A potential contributing factor is the low utilization of health systems, resulting from the perceived low quality of care provided by healthcare personnel. This factor not only frustrates the treatment of endemic diseases but can also hinder the containment of emerging epidemics. Containing epidemics requires adherence to related public health guidelines, such as testing and quarantine. As evidenced by the Covid-19, Zika, and Ebola outbreaks, epidemics and pandemics recur with devastating local and global effects.
In September 2014, when the World Health Organization (WHO) described the Ebola epidemic in West Africa as the most serious acute public health emergency seen in modern times, Sierra Leone, officially the Republic of Sierra Leone, was one of the countries whose chronic health problems were exacerbated by this crisis.
At the end of the crisis, in early 2016, when the Centers for Disease Control and Prevention (CDC) estimated more than 28.000 confirmed, suspected, or probable cases, Sierra Leone accounted for about half of those cases and just under 4.000 deaths.
Intervention Details
Prior to the Ebola outbreak in Sierra Leone, Christensen et al. (2021) designed a large-scale field experiment to evaluate two programs aimed at improving the utilization of government-run clinics and the quality of care provided in those facilities. The research period allowed them to examine the effects of the programs both under “normal conditions” and during the subsequent Ebola crisis (final research was completed in June 2013, and the first Ebola case was reported in May 2014). Subsequently, the authors assessed the effects of these programs during the ensuing Ebola epidemic. This allowed them to observe whether the interventions contributed to the resilience of the health system.
The research included 318 primary healthcare clinics. Of these, 254 clinics were analyzed, with all clinics in the sample separated by at least 3 kilometers to minimize spillover. At the start of the study, the clinics had, on average, just over two staff members present and operated six days a week, serving approximately 450 patients per month.
Over 80% of the clinics had walls and ceilings in good condition, access to running or protected water, and stocks of basic medications (e.g., oral rehydration salts and antibiotics). However, only 10% of them had functional electric lighting.
Methodology Details
Christensen et al. (2021) randomly assigned 254 clinics to one of two interventions or a control group, in partnership with the Government of Sierra Leone and three international Non-Governmental Organizations (NGOs). The first intervention, Community Monitoring (CM), provided information to patients and a public forum to monitor frontline health professionals. The intervention distributed forms to rate local health services and convened interface meetings between community members and health professionals to discuss these ratings and develop “joint action plans” to improve service delivery.
The second intervention provided non-financial awards (NFAs) to improve clinics. Clinical staff were encouraged to develop action plans, and winning clinics received wall plaques and letters of recommendation from the district government. Neither program provided resources to clinics; instead, they aimed to motivate healthcare professionals to provide higher quality care under existing resource constraints.
Aiming to test whether interventions contribute to the resilience of the health system, the study questioned whether they affect the reporting of Ebola cases. To this end, the authors used an unidentified database maintained by the Government of Sierra Leone and the Centers for Disease Control and Prevention to construct weekly counts of tested and confirmed patients in small administrative units called sections. The research focused on the 160 sections that contained a single clinic from the experimental sample, which allowed for the unambiguous coding of the treatment status of each section.
The 254 clinics in the sample were grouped into matched trios using the non-bipartite matching algorithm of Greevy and Beck (2016). Clinics in a trio were located in the same district and exhibited similar levels of utilization and performance at baseline. By blocking matched trios, 84 clinics were randomized to control, 85 to Community Monitoring (CM), and 85 to Non-Financial Awards (NFA).
Results
Two years before the 2014 Ebola outbreak in West Africa, Christensen et al. (2021) randomly assigned two interventions to government-run health clinics in Sierra Leone: one focused on community monitoring and the other providing non-financial rewards to clinic staff. Before the Ebola crisis, both interventions increased clinic utilization and patient satisfaction. Community monitoring also improved child health, resulting in 38% fewer deaths of children under five. Later, during the crisis, the interventions also increased Ebola case reporting by 62%, and community monitoring significantly reduced Ebola-related deaths.
Evidence regarding the mechanisms indicates that both interventions improved the perceived quality of healthcare by encouraging patients to report Ebola symptoms and receive medical care. Improvements in health outcomes under community monitoring suggest that these changes reflect, in part, an increase in the underlying quality of care provided.
Therefore, the study indicates that improvements in the perception of the quality of care in intervention clinics led to an increase in reporting during the crisis, and improvements in the care administered in community monitoring clinics also persisted throughout the crisis period. Thus, community monitoring has qualitatively stronger effects than non-financial rewards to clinic staff before and during the Ebola outbreak. This suggests that engaging the community in promoting accountability can be especially effective in improving the quality of health services.
Lessons in Public Policy
The observed results indicate not only improvements when interventions were implemented in the health system in the short term, but also resilience in relation to crises that occur in the long term. According to Christensen et al. (2021), the increase in patients using health services in the pre-Ebola period, even if not high, has significant effects during the Ebola pandemic.
This suggests that incentives contributed to moderate changes in the perception of quality of care. Therefore, they may strengthen health systems during crises and pay substantial dividends during these critical periods. If these interventions are also effective in other contexts, they may constitute a promising approach to preparing for future health crises.
Improvements in health outcomes under community monitoring indicate that these changes reflect, in part, an increase in the underlying quality of care provided. Overall, the results suggest that promoting accountability not only has the power to improve health systems in normal times, but can also make them more resilient to emerging crises.
References
CHRISTENSEN, Darin et al. Building resilient health systems: Experimental evidence from Sierra Leone and the 2014 Ebola outbreak. The Quarterly Journal of Economics, v. 136, no. 2, p. 1145-1198, 2021.
