Performance Measurements Can Help Programs. It Can Also Hurt Them.

But as football great Rocky Bleier once said, “Preconceived ideas can sometimes become barriers.”  That’s been the unfortunate case with portions of the Temporary Assistance for Needy Families (TANF) program that was passed in 1996 with the intent of overhauling welfare through very flexible block grants to the states. A preconceived idea that there was one be-all end-all performance measurement has thwarted some of the elements in the program’s original vision.

The basic premise behind TANF was that self-sufficiency among recipients of federal aid could be accelerated by mandating work and limiting the time during which cash assistance was available. Setting aside the merit of that policy decision, the approach used to measure “work participation” has been flawed from the beginning, with a single focus on process and compliance, and no accompanying effort to measure results

The Work Participation Rate (WPR) is a calculation that requires states to track the time cash-recipients spend in work or “work activities.” In general, TANF requires 50% of single parents who receive cash benefits to work 30 hours per week (or 20 hours if the child is under the age of six.) If states fail to meet the required rate -- and provide the federal government with documentation of those activities – they can be subject to sanctions that have the potential to reduce their block grant.

The 1996 law gave states a way to reduce the federal 50% requirement. Each single percentage point by which the caseloads drop, allows the states to cut the original 50% benchmark by an equal amount.

But sanctions and incentives can have unintended effects on performance measurement. This is no exception. Many states took actions to create barriers to entry or to impose their own sanctions to families that they deemed uncooperative. Those sanctions have been imposed on black families more frequently than white ones, according to research from the Center on Budget and Policy Priorities

The use of the Work Participation Rate violates lessons that we, and many others, have learned about performance management over the years. We summed up some of the problems in a June 13 Route Fifty column:

“Because the WPR is a process measure without a focus on results, it doesn’t provide information on whether the time spent has an impact on building economic self-sufficiency or increasing wages. By focusing just on work and using a single measure, rather than a basket of measures, it also underplays performance management attention to multiple other family, mental health or crisis-oriented supports offered through the TANF program.”

Some of the flaws in the WPR are very typical of “pitfalls” that we cited in our 2020 book, “The Promises and Pitfalls of Performance Informed Management” and in a recent Urban Institute paper we co-authored with Harry Hatry among others, called “Do’s and Don’ts: Tips for Strengthening Your Performance Management Systems.”

The pitfalls that come with the WPR are not only instructive for TANF reformers, but for practitioners in other fields who can see how bad measures at the wrong level of detail can distort a system.

Listen to Deborah Schlick manager of the Strategic Projects Office in the Economic Systems and Supports division of Minnesota’s Department of Human Services. “The Work Participation Rate doesn’t frankly give us much worthwhile,” she told us. “It’s a process measure that counts how many adults are in officially recognized work activities. The problem is that there’s nothing about the list of activities that research has been able to identify or correlate with employment outcomes or moving out of poverty outcomes.”

The even deeper problem, she said, is that the measurement system saddles caseworkers with a “surveillance function,” rather than a “What-can-I-do-to-help-you function.”

“It poisons the frontline relationship,” Schlick said.

The other problem with the Work Participation Rate is the time it takes away from deeper research that could better inform states and the federal government about how to improve performance.

This not only includes results measurement, but the other vital parts of performance management systems, including the time to collect the data that’s most needed, to ensure data quality, to evaluate programs and to seek evidence-based solutions.

Some states – like Minnesota – have moved into a more results-oriented posture on their own. For years, it has produced reports charting the Minnesota TANF “Self-Support Index”, looking at clients when they enter the program and then checking in three years later to see if they are no longer receiving assistance or, if still receiving assistance, are working at a job at least 30 hours a week.

This work and additional analysis has taught the state a good deal about the range of problems that cause people to need government assistance, about the vast spectrum of needs, about the job history of applicants and the fact that most stay on assistance for only very short periods of time.

A handful of other governments have also developed their own alternative measurement systems. In 2018, the Urban Institute released a report that explored seven states and one city that had established additional performance management systems that went beyond the Work Participation Rate.

But those efforts are still the exception not the rule, and even in those governments, adherence to monitoring the Work Participation Rate leaves less time and energy for results-oriented performance management.

Schlick is excited about what could be learned if performance measures, data analysis and program evaluation could identify the services that were most correlated with successful performance outcomes. But that hasn’t been possible yet. “Good results measurement can allow accountability and flexibility and it can help us figure out where to use flexibility most effectively,” she said. “Someday it would be wonderful to have the chance to do that.”