Speakers
Laurel Eckhouse,
Data and Evaluation Lead, Nava PBC
Michael Chen,
Partnerships and Evaluation Lead, Nava PBC
Ryan Hansz,
Senior Designer/Researcher, Nava PBC
Government teams exploring AI tools are facing growing expectations to demonstrate these tools’ impact on staff efficiency, administrative burden, service delivery, and client outcomes.
This webinar offers a practical, methods-oriented guide to designing real-world impact evaluations for AI systems in public-benefits programs. Drawing from three GenAI pilots led by Nava Labs — including an assistive chatbot, agentic form-filling assistant, and referral generator — we’ll explain how to design data-driven evaluation studies, from randomized experiments with phased rollouts to natural experiments and mixed-methods approaches.
Attendees will see a step-by-step framework for defining key outcomes, selecting comparison groups, implementing randomization, identifying measures, integrating surveys and interviews, and interpreting results responsibly. They’ll also learn the practical constraints of evaluating small-scale pilots with limited sample sizes. Attendees will leave with a toolkit of evaluation models and templates they can use to measure AI’s real impact — beyond usage metrics or anecdotal feedback.
