Next Steps in Moneyball Government

In mid-November, the Government Accountability Office (GAO) released an assessment of how well agencies use performance information in decision making, with a particular focus on program evaluations.  Around the same time, a study on evidence-based policymaking in the states was released by the Pew Charitable Trusts and the MacArthur Foundation; and a new book, Moneyball for Government, by Peter Orzag and Jim Nussle, was announced by the non-profit Results for America.  Finally, a bill was introduced in the House by Congressman Paul Ryan (R), with the support of Senator Patty Murray (D), which would create a commission to recommend ways agencies could better integrate and use existing data, and use program evaluation techniques, to improve decision-making.

The GAO Report.  GAO set out to assess the “extent of agency evaluation capacity” and the progress agencies have made over the past four years, since the GPRA Modernization Act was passed, in improving that capacity.  It also looked for ways agencies have built their evaluation capacity, given that a 2013 GAO survey found that only about a third of program managers reported recent evaluations of their programs.

GAO surveyed the performance improvement officers from the 24 largest agencies and their capacity to conduct and use program evaluation was uneven: “7 of the 24 agencies have central leaders responsible for evaluation; in contrast, 7 agencies reported having no recent evaluations for any of their performance goals.”

GAO observed that an agency’s evaluation capacity depended on both the demand for such information by policy makers, as well as the agency’s ability to supply credible and relevant information.  It says that three elements were present in effective agencies:

• an “enabling environment,” where there was interest in evidence of government performance;
• “organizational resources,” to support the supply and use of credible evaluations; and
• “evaluation results and use” where evaluations were linked to an agency’s management, policy making, and budgeting processes.

An important thread throughout the report was the observation that it wasn’t the supply of performance information that made a difference in its use, but rather the analysis and interpretation of the data for leaders that led to greater use.

In addition, GAO found that only about one quarter of the agencies in its survey “reported that their evaluation offices had a stable source of funding” and concluded that congressional committees could “communicate their interest in evaluation” in a number of ways.  However, it made no specific recommendations.  Interestingly, President Obama recently signed the reauthorization of the Child Care and Development Block Grant, which includes a one-half percent set-aside of funding for program evaluation, so this demonstrates continued congressional interest.

The Pew-MacArthur Study. This succinct guide focuses on what state governments are doing to assimilate performance and evaluation information into decision-making.  It found over the past decade that 100 state laws across 42 states have been adopted that support the use of evidence-based programs and practices. Based on these and other state activities, the guide offers “a framework that governments can follow to build and support a system of evidence-based policymaking.”  Its framework has five components:

• Program assessment. “Government leaders should develop an inventory of programs they currently operate and then assess the available evidence of effectiveness and return on investment for each one.”  For example, Rhode Island developed “an inventory of 58 state-funded programs intended to reduce recidivism in adult and juvenile justice systems.” It found only one-third were not evidence-based and plans to improve its efforts.
• Budget development. “Executive branch agencies should use performance information when developing their budgets” to ensure funding is directed to programs that work.  For example, Connecticut examined its investment in early reading programs and found there was no positive effect on reading skills.  In digging into the data, they found the monies were actually being diverted to other purposes.
• Implementation oversight. “Even the most effectively designed interventions can produce poor results when poorly run,” so the guide recommends strong monitoring systems.  For example, a number of state governments and agencies are using regularly scheduled “PerformanceStat” forums to identify areas for improvement.
• Outcome monitoring. “Routinely measure and report outcome data to determine whether programs are achieving desired results.”  For example, “Virginia Performs” is an interactive, publicly available database that collects and reports performance data on a wide range of government functions at multiple levels” (program, agency, regions, etc.).
• Targeted evaluation. “Conduct rigorous evaluations of new and untested program to ensure they warrant continued funding.” For example, a new “probation with enforcement” program in Hawaii combined data from numerous sources to assess its effectiveness. By linking data from drug test results, offenses, supervision activities, etc,, the program was found to have reduced rearrests by 55 percent.

The guide notes that several federal grant programs have provided incentives for states to invest in evaluation because they targeted about $5.5 billion to seven initiatives that support proven programs.   For example, the federally-funded Maternal and Infant Early Childhood Home Visiting program “requires grantees to direct 75 percent of federal dollars to evidence-based programs and evaluate the impact on key outcomes.” 

The Moneyball Government Book.  Released in mid-November, this book is a compilation of essays by a high-level bipartisan group of advocates for evidence-based decision making, under the aegis of the Results for America advocacy initiative.  According to the press release:

 “This book is about addressing our nation’s great challenges by getting better results with gov­ernment investments. By shifting public resources toward solutions that use data and evidence of impact, we can improve the lives of young people, their families and communities,” said Michele Jolin, managing partner of Results for America. “By bringing together leading thinkers from across the political spectrum and highlighting what is working across the country, this book of­fers a way to open up new opportunities and to make dramatic progress in improving lives. We call upon government leaders at all levels - as well as presidential candidates from both parties - to support and implement the bipartisan ideas in this book.”

Pending Legislation.  In late November, a bill to create a Commission on Evidence-Based Policymaking was introduced by the chairs of the Senate and House Budget Committee chairs – Senator Murray (D) and Congressman Ryan (R).  Champions of the legislation hope that it would be attached to the “must pass” bill to fund the government after the continuing resolution expires on December 11.

According to the bill, the 15-member Commission would offer recommendations on how administrative, tax, and other data collected by the federal government “may be integrated and made available to facilitate program evaluation, policy-relevant research, and cost-benefit analyses.”  It would also assess the feasibility of creating a clearinghouse for agency program and survey data.

So, with all the recent activity on using increased evidence, it will be interesting to see whether the new Congress continues the momentum.

Graphic Credit:  courtesy of Photokanok via FreeDigitalPhotos