Paper Title: Is it time to give up on evidence-based policy? Four answers
Principal investigator: Viviane Pires Ribeiro
Authors: Richard D French
Location of the Intervention: Global
Sample Size: 400 publications
Main theme: Economic Policy and Governance
Main Variable of Interest: Evidence-based policies
Type of Intervention: Literature review on the political potential of evidence-based policies.
Methodology: Qualitative systematic literature review
Given the contradiction or ambiguity in the local and global advocacy of Evidence-Based Policies (EBP), French (2019) conducts a qualitative systematic review of the literature on the political potential of EBP. The study concludes that, to address the challenges faced by EBP, researchers must develop a more realistic understanding of the working environment in which ministers and senior officials operate, reject naive but prevalent assumptions about the level of analytical rationality in government, and recognize that sustained engagement with policymakers may not be compatible with career advancement in academia.
Evaluation Context
The evidence-based policy movement is a sign that major improvements in policymaking are there to be made, but should governments and researchers make the necessary efforts? Are governments currently neglecting evidence that would provide valuable support for significantly improved policies? No one doubts that where research usefully addresses public problems, it should be exploited as much as possible. Not everyone agrees (a) that research 'usefully addresses' public problems, (b) that 'the greatest practicable extent' exceeds the current extent to which research is exploited in policymaking, and therefore (c) that efforts should be devoted to obtaining a better supply of more effectively employed evidence?
French (2019) attempts to answer these questions through a qualitative systematic review of the literature on the political potential of PBE. Thus, the author defines evidence as the product of research: organized knowledge produced according to the standards of relevant academic disciplines; and defines politics as the position or approach adopted by public authorities – governments, agencies, school boards, the military, the police – in relation to problems or opportunities that are perceived as affecting public well-being.
Intervention Details
French (2019) conducts a qualitative systematic review of the literature on the political potential of evidence-based policies. In this sense, the author emphasizes that the literature seeking qualitative systematic reviews should present the following characteristics:
a) Identify the main "schools of thought" in a given area, being alert to the identification of variants, minority and dissenting views.
b) To research across a wide range of disciplines in order to bring different perspectives to the topic at hand.
c) Use complementary electronic and manual search techniques to ensure that materials are not lost due to inadequate indexing or selective database coverage.
However, it is important to note that, just as scientific articles do not provide an account of the research process but rather of its results, policy reports also do not provide any sense of the process of arriving at that policy, with its many reversals, 'irrationalities,' and contingencies. The political rationale accompanying policy announcements should never be confused with accurate accounts of underlying processes or motivations. The point is that, whatever the "educated" layperson decides to assume about public policy formulation, there is no substitute for reading the literature on the subject, also known as "the evidence." Much of this literature argues the bias or unfeasibility of the rationalist model cherished by many proponents of EBP. Similarly, it has been difficult to identify research successes in addressing public policy challenges.
The potential of EBP is therefore contested and controversial. Thus, French (2019) describes four major schools of thought on EBP – the schools Reinforce, Reform, reinvent e Reject.
Methodology Details
The PAIS bibliographic database was consulted for peer-reviewed books and academic articles with the term “evidence-based policy” in their titles. This yielded 132 references, which were supplemented by a bibliography of relevant policy references. Articles that addressed issues using the expression as an indicator of legitimacy, but did not explicitly address the practice of EBP, were discarded from the review. This left several dozen potential references, which were read and from which manual searches of notes yielded several other relevant references. A total of nearly 400 relevant books, book chapters, conference papers, and articles, from a variety of disciplinary traditions, were ultimately identified and reviewed by French (2019).
Results
Based on a systematic analysis of nearly 400 publications, French (2019) identifies four contrasting perspectives on evidence-based policy. A first school of thought advocates strengthening the demands for governments to pay more attention to research. A second perspective advocates reforming the relationships between researchers and policymakers. A third emphasizes the need to reinvent the formal procedures governing the generation and use of evidence. The fourth rejects the possibility that research can simultaneously meet disciplinary standards and meaningfully meet the needs of policymakers.
There is a contradiction or ambiguity in the local and global defense of EBP. It seems that most of this defense betrays a conviction of the obvious merits of the idea, to such an extent that it is itself innocent of much evidence. For the scientifically trained and for social scientists who take their “science” seriously, the idea that policy can significantly benefit from a much greater use of research results seems to be a self-selling idea. There seems to be no need to read the literature on EBP or on public policymaking.
Sir Peter Gluckman (2017), the dean of global scientific consulting, repeatedly warns his audience that scientists should beware of arrogance. It is noteworthy how frequently similar themes – modesty, humility – arise in studies of EBP and related issues. The review located more than a dozen such warnings.
It is unlikely that this situation – a wide range of viewpoints on the viability and imminence of EBP – will change anytime soon. As long as the socialization associated with doctoral work in empirical disciplines remains as it is, there will always be sympathy for rationalist assumptions and perplexity that they no longer seem to apply to the world of practice.
Those in schools Reform e reinvent Those who have devoted significant time and effort to EBP mostly share the conclusion that 'The pursuit of evidence-informed policies and practices will be a long and arduous journey,' and this latter school might even endorse Pawson's statement, despite his own commitment to EBP, 'There is no evidence-based policy. Evidence is the weak point of the political world,' while elsewhere in the academic world and foundations, in the school ReinforceHowever, optimism regarding PBEs remains.
Lessons in Public Policy
French (2019) argues that those who wish to address the challenges faced by evidence-based policies in a fully informed way need to (i) develop a realistic understanding of the working environment of ministers and senior officials, that is, one that goes beyond ritualistic lamenting about their inexplicable failure to heed research and expertise; (ii) address the naive decision theory, grossly overestimating analytical rationality – widely held among academics, researchers, and editors – through the cognitive psychology of decision-making; (iii) given the main conclusion in the EBP literature that “direct and sustained relationships between researchers and policymakers are the ideal method for promoting the use of research in policymaking,” assess to what extent this is compatible with the career demands of 20th-century academia and how its prospects can be enhanced; and finally, in light of the above, (iv) compare the surprising variety of prescriptions from schools of thought. Reform e reinvent and try to reach a realistic consensus.
References
FRENCH, Richard D. Is it time to give up on evidence-based policy? Four answers. Policy & Politics, v. 47, no. 1, p. 151-168, 2019.
