Are Philly-area nonprofits really making a difference? Foundations put up $3 million to find out

Foundations are notorious for demanding that nonprofits document how donations and grants lead to tangible change in the communities they serve, but traditionally they haven’t wanted their money to be used for such monitoring and evaluation.

A $3 million partnership announced Tuesday in Center City aims to help accelerate a shift away from that old model by training Philadelphia-area nonprofits to improve their ability to evaluate what they do and measure the results by using data — skills that are sorely lacking at many small nonprofits.

“It’s encouraging that foundations are putting their money where their mouths are and being true partners when it comes to program evaluation,” said Justin Ennis, executive director of After School Activities Partnerships, which organizes chess, debate, drama, and Scrabble clubs for Philadelphia youths.

The move to measure impact is critical for the Philadelphia region, where tens of millions of dollars are poured every year into nonprofits that seek to improve the lives of area residents.

The new partnership, RISE (Readiness, Implementation, Sustainability for Effectiveness), builds on similar efforts by Philadelphia’s Scattergood Foundation and the Edna McConnell Clark Foundation, of New York, and has the backing of the United Way of Greater Philadelphia & Southern New Jersey and other local foundations.

For the most intensive level of training, RISE will select 15 nonprofits that serve youth and provide the organizations two years of consulting, training, technical assistance, and other support. The program also has a lower tier that will include group training and a small amount of individual consulting to as many as 50 organizations over three years.

The consultants are from the Consultation Center at Yale, which is part of the Yale School of Medicine, in New Haven, Conn.

Yale consultants also worked on the Scattergood Foundation’s Building Evaluation Capacity Initiative, which was started in 2011 with a focus on behavioral health organizations. Participants of that program said it helped them gather evidence on whether what they were doing works.

For example, it’s easy to describe a debate club and make it seem that it’s a good thing, said Ennis, of After School Activities Partnerships, but it’s another to offer evidence. The organization had been conducting before and after surveys, but they were not scientifically validated. A consultant working with Ennis’ group recommended survey software that measures four kinds of student competencies, including sense of belonging and grit, to assess the program’s before and after impact.

Maddy Booth, education program director at the Vetri Community Partnership, said the Scattergood program prompted the organization to improve the follow-up to its Mobile Teaching Kitchen, which offers cooking classes and demonstrations at schools, community events, and farmers markets. Vetri started checking in with participants a month after they visited the Mobile Teaching Kitchen to see what difference it had made and what the barriers were to changing cooking habits.

Hopeworks, a Camden nonprofit that trains youths in website development and database coding, has shifted its focus to long-term impacts, said Valerie Buickerwood, the organization’s director of engagement and communications.

“We started to really track alumni to see the long-term impacts of our program. Are our youths getting full-time jobs when they are finished? If so, how long are they keeping that job? Are they moving up? Are they getting raises? Are they being promoted? If they are not, then we know that along the line we missed preparing them for something,” she said.