EES 2023: Sharing our journey on AI’s application in qualitative research

Hi there, Steve and I (Gabriele) are excited to tell you about our latest work on using AI in qualitative research.

Back in November 2023, Steve took the stage at the European Evaluation Society. His presentation, titled "What Influences What? Using AI to Turn Stakeholders' Stories Into Causal Maps, Rapidly, Rigorously and at Scale", for us it wasn't just any talk; it was a landmark moment in our work with AI in evaluation.

The presentation is accompanied by a report detailing our methodology with a proof-of-concept research study.

What we find so exciting is that how the causal mapping approach provides such a simple way to make sense of large amounts of text. Basically, you make a high-level statement explaining any particular aspects you are interested in, press the big green button, (cross your fingers) and watch the maps appear. It’s a very different feel from working with traditional qualitative data analysis.

Our approach introduces two innovative uses of AI in evaluation:

1. Automated Qualitative Interviews: conducting extensive, AI-powered online interviews to probe (in this study case) current issues in the USA. 2. Automated Data Analysis: using AI to automatically code and analyse these interviews, creating detailed causal maps that reveal shared perceptions, group-specific views, and evolving trends.

It's been an adventure, figuring out how these AI tools can help evaluators get a grasp on the complex systems they work with - projects, programs, or policies.

If you want to know more about this, check out our full report here. We'd love to hear your thoughts!