How Will Government Adapt?: Finding and Funding What Works

This is the third blog post in a series that sums up highlights of sessions held as part of the annual meeting in mid-November of the National Academy of Pubic Administration. The moderator for this session was Jon Baron, President, Coalition for Evidence-Based Policy. Panelists Grover J. “Russ”Whitehurst, Director, Brown Center on Education Policy, Brookings Institution; Former Director, Institute of Education Sciences, U.S. Department of Education Naomi Goldstein, Director, Office of Planning, Research and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services Demetra Nightingale, Chief Evaluation Officer, Department of Labor Highlights Background. This panel explored the efforts to increase government effectiveness through the use of rigorous evidence about what works. In most areas of social policy – such as education, poverty reduction, and crime prevention – government programs often are implemented with little regard to evidence, costing billions of dollars without meeting critical outcomes or addressing serious social challenges. However, studies have identified potential for highly-effective program models and strategies, illustrating that a concerted government effort to integrate these interventions can bring rapid progress to social policy. The panel discussed the role that evidence-based policy has played in three leading social spending agencies, stressing the importance of not only incorporating the identification of “what works” into the process but to ensure that evidence is a criteria for allocating funds so that those programs that are proven to work get priority. The facilitated discussion addressed concerns that policy decisions are not being made on the basis of evidence and panelists shared some leading practices for how to “move the needle” in this promising area. Some Key Insights Raised: Tools such as the What Works Clearinghouse and the Top Tier Evidence Initiative have enabled users to improve their ability to achieve desired outcomes and make data-based decisions for funding. Key components of an effective evaluation process include rigor, relevance, transparency, independence, and ethics. The Random Control Trial methodology is effective but not the only rigorous evaluation – other formats (such as rapid cycle evaluation) should be constantly explored. Discussions about supply and demand of evidence continue – the supply (and quality of the evidence) is increasing but the demand for evidence-based approaches has lagged in certain areas. Important to focus on the questions being asked, the partnerships developed in the process, and the dissemination of the work (availability of data to others.) The sophistication of the evaluation process, as well as its integration with Government Performance and Results Modernization Act initiatives vary across agencies and departments. Evidence-based initiatives should not be a way to eliminate programs but they should also help to improve program outcomes. An open environment that supports risk taking and embraces innovation is essential to effective evidence-based decision making. Important to note difference between impact and outcome – there can be a negative impact in an experimental design, but that does not necessarily result in a negative outcome for the citizen. Evidence and evaluation offices within federal agencies are becoming more involved in strategic plan development and budget submission processes. (summary by Brenna Isman, NAPA) Visit the landing page of this initiative where you can learn of future blog post topics and follow previous conversations.