Building a Culture of Informed Decision Making

A recent Government Executive article by Ed O’Brien, an associate professor at the University of Chicago’s Booth School of Business, observes that “people assume they can and will use more information to make their decisions than they actually do, according to the research.”

While this is distressing news to those who promote the use of evidence in decision making, it doesn’t have to be that way. There is an effort to actually change the culture within federal agencies to become more evidence-based in their decision making.  The recently signed Foundations for Evidence-Based Policymaking Act of 2018 creates new momentum for this culture change.

The New Evidence Act. This new law has three distinct sections:

• The first section creates a new capacity for, and governance of, evidence building activities and the use of data. For example, it mandates agencies to designate an agency evaluation officer, along with the creation of a career path and/or professional occupational job series for evaluators. It also requires agencies to develop evidence-building plans (sometimes called “learning agendas”) that address key policy questions that policymakers want answered.
• The second section focuses on increasing access to government data.It requires agencies to designate chief data officers, conduct inventories of the data they have, and generally make their data “open by default.”
• The third section strengthens privacy and confidentiality of agency data by codifying several existing administrative practices to protect against inappropriate disclosure of personal information. But it also encourages data sharing by shifting assumptions from “no access without explicit authorization” to a “presumption of access unless prohibited.”

Shortly after the Act was signed, a discussion panel hosted by the National Academy of Public Administration met to discuss the implementation of the new law. Following are highlights of that conversation.

Key Implementation Challenges. Of course, the primary locus of implementation of the new law will be at the agency and program levels, but having the top institutions aligned will be important to create sustainable implementation over a period of years. Experts highlight two key challenges in the initial implementation of the Act:

• The first is identifying the big policy questions that need to be asked by agencies.Agencies have to define what they want to know and where the data is located that can answer these questions. The existing annual reviews of agency strategic objectives could be one venue for identifying questions, but some of the questions will reach across agency boundaries and will need cross-agency advocates to raise and answer them.
• A second challenge is coordinating implementation across the various cross-agency management councils (such as the President’s Management Council, the Program Improvement Council, the Chief Information Officers Council, etc.) and program-level executives – both in OMB as well as within the agencies. Typically, each has different perspectives and portfolios as to what the priorities should be for a common research agenda. Historically, there has been too much infighting over the control of data and turf to craft such agendas and a new governance structure will need to be created at multiple levels in order to provide needed coordination.

Three Parallel Implementation Strands.  To align the top-level institutions of government around a common agenda and to create an evidence-based decision-making culture, there are three paths to strategically ensure the implementation of the new law focuses on culture change and not become an exercise in compliance with new statutory requirements:

• Office of Management and Budget.  A key role for OMB will be to connect people within agencies across their traditional silos and help ensure the agencies ask questions that matter. OMB is in a position to play a catalytic role in creating a new learning culture and the necessary supporting systems required to develop and bring evidence to bear at all stages of policy development and implementation. 

For example, OMB could use the currently-ongoing development of the Federal Data Strategy to help bridge institutional silos. OMB has done this successfully in the past; the Bush Administration’s Program Assessment Rating Tool was a good example of how to bridge across the internal OMB silos.

• Congress. It would be particularly helpful if Congress were to offer a reinforcing message, such as via appropriations committee report language.  Also, authorizing committees could ask their agencies how they are identifying the most important questions that the agencies need to know answers for.  The Congress needs to develop a process for engagement, especially since a number of different committees have jurisdiction over different parts of some agencies. This has precedent. A similar coordination effort was undertaken in the mid-1990s in the House via a set of special cross-committee task forces that assessed agencies’ first strategic plans developed under the Government Performance and Results Act.
• Outside Stakeholders. State and local governments could help agencies formulate mission-oriented research questions; academics and research community could focus on statistics and evaluation techniques; and private industry could focus on business-related interests.  Advocacy groups have a role as well, providing outside pressure to help sustain momentum (this happened with the implementation of the DATA Act). 

Anchoring the Implementation Within Agencies. The agencies are where the work must be done to ensure the legislation does not become just another set of requirements to implement. If OMB and Congress truly want to institutionalize an evidence-driven culture in agencies, efforts will be needed to connect evidence and data to the management of actual programs, including at the state and local levels. A key piece of this will be the development of agency-level data governance that defines and leverages the roles of the newly-designated agency evaluation officers and chief data officers.

One approach might be for a third party to create a public scorecard for each agency to assess progress and allow agencies to compare themselves with their peers.  The right metrics would capture not just procedural changes but also the successful application of evidence to policy. A possible example is a scorecard developed by the nonprofit Results for America. In that scorecard, the Department of Labor is one of the pioneering agencies that has made significant progress in embedding the use of evaluation and evidence into its decision-making process. Scorecards have been used in other program areas to draw attention to an initiative and create a sense of urgency for continued implementation momentum.

Graphic Credit: Courtesy of bigjom via FreeDigitalPhotos.net