training

Evaluation Workshop with Rebekah Levin

Evaluation.

No word provokes fear more promptly among non-profit organizations, who depend on positive assessments of their work to keep receiving grants, contracts and other crucial resources. But evaluation can help grassroots leaders do their work and communicate effectively with funders as well as potential supporters, according to Rebekah Levin, director of evaluation and learning with the Chicago-based Robert R. McCormick Foundation, which makes about $80 million in grants each year.

On March 29, Rebekah led a virtual workshop for an international roster of Chicago Peace Fellows and alumni of GATHER, the Goldin Institute’s on-line capacity-building curriculum. With 3 decades of experience as an evaluator, an academic and a political activist herself, she has become a self-described “proselytizer for evaluation,” especially when it is practiced in a way that empowers community leaders rather than penalizes them for not fitting into narrow categories and pre-ordained expectations.

[quote]“Though there are horrible things that have come from evaluation, there is fantastic stuff that has come from it, and when you don’t use it, I tell people in activism that it’s like leaving money on the table. It is a very powerful tool and you should have it working for you.” -- Rebekah Levin [/quote]

Philanthropies, government agencies and other funders demand data and have a strong preference for numbers and quantitative information over qualitative information and anecdotes, while generally failing to account for their own biases and assumptions. People who work for non-profit groups, in turn, frequently find evaluation to be a burden or a problem, and begin their evaluation process by asking “What data can I get?”

However, Rebekah averred that grassroots organizations can take control of the process by defining the questions that will give them information to help them do their work more effectively.

“With evaluation, you are gathering data to answer a question for which you don’t have the answer,” she said.

[quote]“Gather your data to answer questions, and when you have the answers to those questions, use those data to teach your supporters or people you want to support your organization about the power of what you’re doing.”[/quote]

The Chicago Peace Fellows and the alumni of GATHER’s inaugural class operate grassroots operations in a wide variety of circumstances, but all have extensive experience with evaluation, and they were brimming with questions for Rebekah. Eyob Yishak, a peace program coordinator from Addis Ababa, Ethiopia, said his program spent a year coming up with evaluation parameters and assigned one person with monitoring responsibility, but after hearing Rebekah speak, he wondered aloud if evaluation should be a team effort.

Speaking as an evaluator herself, Rebekah agreed with Eyob’s instinct, and encouraged those with evaluation duties to fully involve those who were doing the work as well as those being studied throughout the process.

“It’s a humbling thing,” Rebekah confessed. “We have to let go of our power and our roles.”

Cynthia Austin, who works with survivors of sexual violence from her base in San Diego, USA, said her team produces a lot of data about age and demographics, especially during their intake of women leaving the sex industry, but aren’t always sure what they can do with it.

Rebekah encouraged Cynthia’s team and other groups in that situation to reflect on their work, introspectively consider what information they really need, and improve their efforts by adding additional data to those which do prove useful.

Diana Alaroker, the accounts manager and social worker at Youth Leaders for Reconciliation, Education and Development in Gulu, Uganda, noted that many international agencies paid people for interviews, which risked the integrity of the research.

[quote]“When you give them the money, they give you the answers you want to know, not what they have really experienced.” -- Diana Alaroker[/quote]

The phenomenon Diana described was widespread, Rebekah said, recommending that the subjects of the interviews be fully involved so that they understand the potential benefits of accurate research to them and their community.

“Bringing people into the power circle can really have an effect,” she added. “You can use this as a tool to make the evaluation stronger.”

Peace Fellow Robert Biekman, senior pastor of Maple Park United Methodist Church and urban ministry coordinator of the Northern Illinois Conference of the United Methodist Church, wanted to know how small organizations could handle the cost of evaluation.

Rebekah answered that funders had an “ethical responsibility” to pay for the costs of any required data collection, but she always tried to match the methodology to the available resources:

[quote]“To me, it goes back to if the question is really important, what will it worth to you to get it answered?”

Nevertheless, Rebekah quickly added that evaluation doesn’t have to cost extra money and can be done less formally. Basic evaluation can be performed even if organizations sit together at regular intervals, talk about what they’re experiencing, take notes, and then analyze those records.

“Like anything in your life, you should be asking questions and looking for information that helps you learn and move forward,” she said.

Frank Latin, founding executive director of the Westside Media Project, said his organization already receives funding from the McCormick Foundation and others, but because they were concentrated in schools in one Chicago neighborhood, felt that he needed to use evaluation to explain he was having a significant impact.

Rebekah urged Frank to collect both quantitative and qualitative data, and use stories to tie them together and let funders understand the breadth as well as the depth of his organization’s work.

Jamila Trimuel, founder of Ladies of Virtue, an award-winning mentoring and leadership program on Chicago’s South Side, wanted to know how to approach universities to do evaluation.

Rebekah cautioned that while working with academics is appealing because they are thorough, professional and often do not charge, non-profits should make sure that their academic partners’ agendas align with theirs.

“Some of the university research is very good, but some of it is not benefiting communities,” she explained.

The online workshop ended with applause from the Fellows in Chicago and around the world and a commitment to continue the conversation about how evaluation can help us improve our efforts towards community driven social change.

by Ethan Michaeli