Your podcast has listeners. But does it have impact?

Imagine a podcast production team equipped with an analytics dashboard and a slew of metrics: Downloads, perhaps broken down by geography; plays; listeners; engaged listeners. These metrics can provide some insights, but they can’t by themselves answer the questions that matter most to the people making the podcast: Is the podcast making a difference?

To answer such a question, we can turn to evaluation. It’s first necessary to move beyond analytics and start asking learning questions — the type Impact Architects encourages many of its partners to ask. In early 2025, the Center for Effective Philanthropy engaged Impact Architects to assess the impact of their Giving Done Right podcast. The podcast aims to provide listeners tools to make an impact with their charitable giving. The Center for Effective Philanthropy had the analytics identified above (which are useful, especially in context of other data), but they were curious about how — ahem — effective the podcast truly is. Who is actually listening? Why are they listening? What do they find useful about the podcast, and what are they doing with the information? What would make the podcast even more useful in the future? 

To answer these questions, we engaged with listeners directly. In collaboration with the Center for Effective Philanthropy, we developed a listener survey to collect broad information, an interview protocol for in-depth insight, and a list of listeners for one-on-one conversations, intentionally created to hear diverse perspectives. The feedback was rich, and it provided the depth of insight that we could not get through analytics alone. For example, we were able to go beyond “which episodes had the most downloads?” to learning which themes and episodes were most and least useful, and why.

That is not to say that there’s no need for podcast analytics. In fact, some of the most potentially useful insights emerged when we analyzed the qualitative data that came directly from listeners alongside the quantitative data the Center for Effective Philanthropy already had on hand (along with some creative analysis of podcast data). For instance, some of the themes interviewees said were most useful were sometimes but not always contained in the most listened to episodes. We had to ask: Why, and what does that mean for the future of the podcast?    

The final piece of the puzzle was sharing our findings of all of the data with the Center for Effective Philanthropy over the course of the project. The best insights don’t just come from analyzing all of the data available — whether existing quantitative data or qualitative data we need to gather ourselves — but by sharing what we’re learning long before the final report. 

We learned a great deal about what makes the Giving Done Right podcast meaningful for its listeners — practical information and models that can be applied in multiple contexts — as well as what can be tweaked to make it even more impactful — foregrounding that actionable information in titles and descriptions, somewhat shorter episodes, a broader set of guests and roles. With Season 6 of the podcast announced, we look forward to seeing (and hearing) the applied learnings.   

The mix of quantitative and qualitative data, together with deep engagement with the Center for Effective Philanthropy, that led us to our insights reflects how Impact Architects supports our partners in learning and evaluation. While audience analytics are helpful for day-to-day insights, bigger picture learning and evaluation require deeper research and direct engagement. If you’re a podcast producer or otherwise create content in which you have that lingering question — is what I’m doing making a difference? — evaluation and learning may be what you need to find out.