by Louisa Hackett - Recently, a few community-based organizations, evaluators and funders gathered at the City University of New York Graduate Center to discuss, as one participant described it, the accountability regime eating us.
In discussing how evaluation is affecting (or as some might say infecting) the nonprofit sector, speakers mentioned all the research studies that report the same obvious and well known findings: lack of childcare, healthcare, access to education, employment and transportation restrict vulnerable populations’ ability to move out of poverty. And, even more distressing, is that funding for evaluation and research undercuts the money available for programs.
A recent case in point is the New York City-sponsored Opportunity NYC Family rewards that paid poor people to encourage good behaviors (going to the dentist; attending school regularly) and self-sufficiency (holding down a full-time job; passing the high school Regents exam). The program which paid out more than $14 million to over 2,500 families also cost $10.2 million to run and $9.6 million to evaluate according to the New York Times.
Government and foundations interest in funding ‘evidence-based’ programs is founded on a belief that scientifically designed research methods reveal what works. However, as the group explained, complex social problems require complex interventions and complexity can not always be easily studied like a science experiment.
Accountability does matter. Collecting data has the potential to improve programs. The question becomes creating useful evaluations or feedback mechanisms to make sure programs are effective and bad programs can be improved. As Katya Fels Smyth and Lisbeth B. Schorr point out in their paper “A Lot to Lose: A Call to Rethink What Constitutes ‘Evidence’ in Finding Social Interventions that Work,” by basing our judgments on many ways of knowing and many sources of evidence, we can avoid the false choice between relying on random assignment experiments versus relying on professions of good intentions, ideology, and a handful of anecdotes.”
How can a nonprofit respond and be equipped to explain their programs impact? One way is for groups to take back the evaluation language and explain to funders this is how we define and measure success. An initial step is to develop a theory of change that makes explicit the link between a program’s activities and the impact it wants to make. Another step is to gather information, including stories that indicate progress is being made within the program’s identified domains of impact. And finally, groups can say to those funders requesting evaluation data, “will you support our ability to gather and report evidence our program works over and above the funding for the program?”
Subscribe via RSS feed 
Comments
Allison
For the past few years, I've been trying to get nonprofits and funders to understand the difference between the terms accountability and evaluation. Accountability is basically the process of determinig if a nonprofit is behaving properly - are they spending money responsibly, are they providing the services they proposed, and are they serving the individuals they are supposed to.
Evaluation is really the process of determining what outcomes resulted from a program, sevice, and project, as well as determining what components lead to these outcomes. In it's most useful sense, this evaluation data would then be used to inform the next evolution of the program, sevice, or project.
Unfortunately, both funders and nonprofits have been stuck in thinking of only accountability. Blame falls on both sides.
Funders assume most nonprofits aren't sophisticated to do evaluation, and the funders rarely provide the additional funds necessary to do evaluation. Therefore, only accountability measures are requested as part of a reporting process.
Nonprofits also fail to stand up to funders and request additional funds for evaluation. Nonprofits too often fall into thinking that they can't question the funder, which leads to just doing the bare minimum that the funder wants, even if that limits the amount of knowledge the nonprofit could generate from doing evaluation (and not just accountability).
As for your idea that nonprofits need to redefine programs, Here Here!! See similar musing "Hope, dignity and quality of life are also valuable outcomes, even when measured in hours." at http://bit.ly/8iYuEk Gayle