📚 Grantee Report Analysis · ITAD · Multi‑AI Tools · 2026

Grantee Report Analysis and Synthesis Using Multiple AI Tools: An ITAD Case Study

Grantee report analysis and synthesis using multiple AI tools refers to ITAD's deployment of Copilot, ChatGPT, and MaxQDA (with Tailwind AI) to evaluate a partner organisation supporting African evidence‑generating institutions. Each tool handled distinct tasks—framework development, contextual analysis, and qualitative synthesis—demonstrating that specialised AI tools, when embedded in human‑led workflows, can efficiently process large document sets while maintaining evaluative rigour.

🏛️ Evaluator: ITAD 🌍 Focus: African evidence institutions 🤖 Tools: Copilot · ChatGPT · MaxQDA (Tailwind AI)

⬇️ Read the original case study (PDF) – ITAD guide on AI

What is the role of multiple AI tools in grantee report analysis?

In monitoring and evaluation, using multiple AI tools allows teams to match specific technical capabilities to distinct analytical tasks. Rather than relying on a single platform, evaluators can deploy specialised tools for framework development, large‑scale document synthesis, contextual country analysis, and qualitative coding. This modular approach—exemplified by ITAD’s evaluation—maximises efficiency while preserving the centrality of human interpretation.

Which program and partner organisation did ITAD evaluate?

ITAD conducted an evaluation of a partner organisation that supports African institutions generating evidence and data. The initiative’s goals were twofold:

Strengthen local evidence production capacity
Improve policymakers’ ability to use evidence in decision‑making

The evaluation assessed the value contributed by the broader evidence ecosystem, the partner organisation’s specific role, and possible responses to future developments in the sector. With approximately 450 grantee documents to analyse—plus key informant interviews and focus group discussions—the team faced a substantial qualitative data challenge.

Which AI tools were used, and for what specific purposes?

💬 Copilot

Analytical framework development · Secure document synthesis · Pattern/gap identification across 450 grantee documents

🤖 ChatGPT

High‑level contextual analysis for three country case studies

📊 MaxQDA + Tailwind AI

Synthesis of coded qualitative data from KIIs and FGDs, integrated with manual coding

How did Copilot support the evaluation?

📌 Copilot’s role: The team used Copilot (with its Notebook feature) to store evaluation context, develop the analytical framework, and extract evidence from ~450 grantee documents. It then synthesised findings by grantee and by country, helping to identify patterns, gaps, and cross‑cutting themes.

Copilot’s ability to securely process large volumes of unstructured text within a bounded framework made it ideal for this foundational synthesis work.

How was ChatGPT integrated into the workflow?

📌 ChatGPT’s role: ChatGPT was used for high‑level contextual analysis in three country case studies. It helped the team rapidly understand each country’s evidence ecosystem, policy landscape, and institutional dynamics—providing background that informed deeper qualitative inquiry.

What role did MaxQDA (Tailwind AI) play?

📌 MaxQDA + Tailwind’s role: After manually coding key informant interviews and focus group discussions, the team used MaxQDA’s Tailwind AI feature to synthesise the coded excerpts. This generated thematic summaries that remained grounded in human‑defined categories.

What was the workflow that enabled good results?

🧩 1. Different tools for different tasks

Copilot for secure synthesis, ChatGPT for contextual analysis, MaxQDA for coded data—each deployed where it added most value.

👁️ 2. Human supervision at every stage

AI tools were used as supporting instruments, not decision‑makers. All outputs were reviewed, interpreted, and triangulated by evaluators.

⚙️ 3. Integration with manual coding

MaxQDA’s AI worked on pre‑coded data, ensuring that synthesis remained anchored in human‑derived categories.

📁 4. Copilot Notebook for context retention

Storing evaluation context in Copilot’s Notebook reduced prompt drift and improved consistency across extractions.

What limitations and challenges emerged?

⚖️ Weakness in judgement	AI outputs were stronger when synthesising pre‑analysed data than when performing standalone analysis. Direct AI coding was not used.
🔍 Pattern over‑identification	AI tools tended to detect too many patterns, some spurious. MaxQDA’s automated AI coding was therefore not used in the final analysis.
📄 Formatting weaknesses	Copilot often produced outputs not in the required format for Excel coding, requiring analysts to manually correct and restructure.

How did the team mitigate these limitations?

Human judgement first: All AI outputs were treated as drafts, reviewed and refined by evaluators.
Selective use of AI coding: Automated AI coding in MaxQDA was discarded; instead, AI was applied only to manually coded excerpts.
Prompt engineering and context storage: Copilot Notebook helped maintain consistency and reduce formatting issues, though manual correction was still needed.
Triangulation: AI‑generated patterns were cross‑checked against interview data and document review to filter over‑identification.

What are the key lessons for evaluators using multiple AI tools?

✅ Match tool to task

Copilot for synthesis, ChatGPT for context, MaxQDA for coded data—specialisation matters.

⚠️ AI struggles with judgement

Do not rely on AI for standalone interpretation; use it on pre‑analysed material.

🔎 Pattern inflation is real

AI sees patterns everywhere—human review must filter.

📏 Formatting friction

Budget time for correcting AI outputs into usable formats (e.g., Excel).

🧠 Human in the middle

AI works best embedded in human workflows, not as an autonomous analyst.

Frequently asked questions

What is grantee report analysis using multiple AI tools?

It is the practice of deploying different AI platforms for distinct analytical tasks within a single evaluation. For example, using Copilot for secure document synthesis, ChatGPT for contextual background, and MaxQDA for synthesising coded qualitative data—all under human supervision.

Which AI tools did ITAD use in this evaluation?

ITAD used Copilot (framework development, document synthesis), ChatGPT (country case study contextual analysis), and MaxQDA with Tailwind AI (synthesis of manually coded KIIs and FGDs).

What were the main limitations of using multiple AI tools?

Key limitations included AI’s weakness in standalone judgement, a tendency to over‑identify patterns (leading to spurious themes), and formatting inconsistencies (e.g., Copilot outputs not matching Excel requirements).

Can this multi‑tool approach be replicated by other evaluators?

Yes. The principle is to match each tool’s strengths to specific sub‑tasks, maintain human oversight, and be prepared to iterate. Teams should budget time for output formatting and pattern verification.

Main reference & original source

📘 This case study is based on the official ITAD guide on artificial intelligence in evaluation, published in March 2026.
Source: ITAD guide on AI (PDF) – EvalCommunity repository

🔗 Access full ITAD case study (PDF)

⚖️ Interpretation and analysis by EvalCommunity: The insights, lessons, and expert commentary presented in this article are offered by EvalCommunity as an independent interpretation of the ITAD case study. They are based on the information published in the ITAD guide on AI (March 2026) and are intended to support learning within the M&E community. EvalCommunity does not claim ownership of ITAD’s methodologies or findings, and this content should not be read as an endorsement or official representation of ITAD’s work. We encourage readers to consult the original PDF for the complete context.

Resources for further learning

Master multi‑tool AI workflows in M&E

Learn how to combine Copilot, ChatGPT, and qualitative analysis tools in real‑world evaluations. Access courses, templates, and expert guidance.

EvalCommunity Services AI in M&E Course

Grantee Report Analysis and Synthesis Using Multiple AI Tools – Case Study