When Bigger IS Better

This team of registered nurses is part of a national network created by the Nurse-Family Partnership -- a 40-year old non-profit that visits the homes of low-income, first-time mothers to help “transform the lives of vulnerable, low-income mothers pregnant with their first child.”

The visits begin during pregnancy and continue for two years following birth. The Nurse-Family Partnership is an evidence-based health program that has helped over 300,000 first-time mothers since its founding. Randomized control trials that have evaluated the effectiveness of this program demonstrate that it results in a 48 percent reduction in child abuse and neglect, a 56 percent reduction in emergency room visits for accidents and poisonings, and over time a 59 percent reduction in child arrests at age 15.

Today, this program is in about 270 locations in 41 states around the country, serving 55,000 families a year.  But the need is much greater.  Shouldn’t programs like this be scaled to cover more in need? And if so, how? A new IBM Center report by Patrick Lester, director of the non-profit Social Innovation Research Center, explores these questions and offers recommendations for action.

Background.  Government is now at a point where its support of evidence-based programs is moving from being seen as a shiny new boutique approach – such as the i3 education programs or Pay for Success initiatives -- to being embedded in more established, multi-billion dollar federal social service entitlement programs.

However, for many of these larger social service programs, identifying and supporting evidence-based programs is a new territory for those implementing these programs on the front line.  This is why understanding how to scale evidence-based programs has become progressively important for federal agencies, and state and local service providers.

Entire sectors of government programs are now being forced to transform how they do business as Congress increasingly requires that federally-funded grant programs be rooted in evidence that demonstrates that they are effective.

For example, in early 2018, President Trump signed into law the landmark bipartisan Family First Prevention Services Act. This law reforms a set of existing efforts that help keep children safely with their families and avoid the traumatic experience of entering foster care. Part of the new law incorporates a requirement that funded programs demonstrate that they are evidence-based. This is a significant departure from past federal policy that left such decisions to the states. Programs that qualify as “evidence-based” have to be approved by the newly-created federal Title IV-E Prevention Services (Family First) Clearinghouse.  In addition, states must submit five-year plans for review and approval by the federal government, as well. As of January 1, 2020, only Utah and the District of Columbia have had their plans approved.

Barriers to Scaling Up Successful Programs.  The Nurse-Family Partnership is somewhat of an anomaly in that its success has been independently validated. Lester identified a number of barriers to scaling up successful programs. For example, he found that most child welfare interventions have not been sufficiently validated by rigorous studies. Of the 482 programs catalogued by the California Evidence-Based Clearinghouse for Child Welfare, only 35 (seven percent) meet its criteria for being well supported by research.  In addition, even when research does exist that identifies successful programs, researchers and practitioners note that it takes an average of 17 years for research results to find their way into practice.

But scaling successful programs matters. One review of nearly 500 child and adolescent programs found that those that were well-implemented produced effects that were two to three times greater than those that were not.

How can these barriers be overcome? What does it take to scale evidence-based programs successfully?  Lester examines these questions within the context of the Family First Prevention Services Act.

State Efforts Are Key to Scaling Social Programs.  Lester’s research found that state-level efforts may be the key to successfully scaling effective, evidence-based social programs.

First, successful scaling of such interventions appears to require active and targeted support from government agencies. Typically, they are the ones who have the financial resources and legal authority necessary to scale public programs significantly.

Second, while public policy decisions may be the critical drivers of scale, the key ingredient to effective scaling appears to be a supportive infrastructure that assures quality and fidelity to core components.

Federal laws like the 2010 Maternal, Infant, and Early Childhood Home Visiting program and the 2018 Family First Act are helping to scale evidence-based interventions, but the federal government is not acting alone. States are also playing a leading role. Lester writes that some of their most promising strategies include:

• Including Evidence Reviews in State Budgeting Processes. Some states now incorporate evidence reviews in their budget processes. For example, Colorado requires state agencies that request new funding to summarize and cite the research on program outcomes and to conduct a cost-benefit analysis. Minnesota and New Mexico have instituted similar requirements.
• Mandating Investments in Evidence. Some agencies have mandatory evidence targets established by state law that require them to spend a certain percentage of their funding on evidence-based programs. For example, Oregon passed a law in 2003 directing five state agencies to spend at least 25 percent of their funds on evidence-based programs by 2007. The threshold rose steadily thereafter, reaching 75 percent in 2011.
• Leveraging Existing Medicaid Spending Flexibilities. Medicaid is a substantial source of funding for many child welfare-related services. For example, South Carolina has used its Medicaid waiver authority to establish a pay-for-success program that has expanded Nurse-Family Partnership services in the state.
• Including Contract and Grant Requirements to Adopt Evidence-Based Approaches. Some state and local agencies direct providers to adopt evidence-based interventions through contract and grant requirements. Performance-based contracts, which are common in many states, may also help scale evidence-based interventions.
• Offering Training and Technical Assistance for Evidence-Based Approaches: Some states fund programs that defray the costs of training and technical assistance for evidence-based programs. Examples include the Child Health and Development Institute of Connecticut and the Evidence-based  Prevention & Intervention Support Center in Pennsylvania.

The Federal Role.  While states are on the front line for scaling many social programs, there is a significant federal role. For example, the Evidence Act requires federal agencies to create “learning agendas” to set priorities for evidence building.  These agendas will highlight federal investments in research.  Also, federal programs are increasingly adopting “tiered evidence” approaches to funding, which over time should better target funding when scaling programs.