AI Research Protocols for M&E and Development Professionals

A practical guide to using notebook-based AI for evaluation, evidence synthesis, programme learning, research, reporting, and development decision-making without allowing speed to replace professional judgement.

1. Why M&E Professionals Need Research Protocols for AI

AI notebook tools can make it much easier to work with evaluation reports, research papers, programme documents, monitoring documentation, interview materials, policy papers, and other sources.

But a polished answer is not automatically a defensible answer. An AI system can produce a confident, well-written synthesis that is problematic because the underlying research workflow was weak.

For example, you may have:

• asked an ambiguous evaluation question;
• combined sources with different definitions or reporting periods;
• treated repeated claims as independent evidence;
• mixed primary evidence with secondary commentary;
• uploaded several versions of the same report;
• allowed an AI-generated summary to become a de facto source;
• accepted an interpretation because it confirmed an existing assumption.

These are not necessarily hallucination problems. They are research process problems.

2. Prompt vs Protocol

A conventional AI workflow often looks like this:

Upload reports
      ↓
Ask a question
      ↓
Receive a summary
      ↓
Use the summary

A protocol-based workflow adds deliberate checks:

Define the question
      ↓
Architect the source set
      ↓
Check source quality and independence
      ↓
Analyse claims and evidence
      ↓
Search for contradictions
      ↓
Test alternative explanations
      ↓
Stress-test the conclusion
      ↓
Review uncertainty and provenance
      ↓
Human judgement
      ↓
Use or publish the result

That extra friction is useful in evaluation and development work because it creates opportunities to catch weak assumptions before they become findings, recommendations, or decisions.

3. Protocol 1: Deep Drill

M&E example

Instead of:

Summarise whether the programme was effective.

use a structured request:

Identify the main effectiveness claim.
Trace the evidence presented for that claim.
Identify methodological assumptions and limitations.
Identify qualifications and competing explanations.
State what the source does not establish.

4. Protocol 2: Radar

Ask the AI to classify the material into:

• Primary evidence: direct data, evaluation evidence, documented observations, or original research.
• Secondary interpretation: claims or conclusions drawn from underlying evidence.
• Repeated claims: statements appearing across several sources.
• Independent corroboration: claims supported independently rather than merely repeated.
• Contradictions: sources that disagree in definitions, results, or interpretation.
• Evidence gaps: questions that cannot be answered from the available material.

5. Protocol 3: Crucible

Example: avoiding an unsupported causal claim

Initial statement: “The programme improved employment outcomes.”

A Crucible review might ask:

• Is there a credible counterfactual?
• Are baseline and follow-up populations comparable?
• Could labour-market changes explain the result?
• Was exposure to the intervention consistent?
• Does the evaluation design support a causal conclusion?

The resulting statement might instead be: “Employment outcomes increased between baseline and follow-up among programme participants, although the available evidence does not establish that the programme alone caused the change.”

6. Protocol 4: Control Problem

7. Protocol 5: Architect

Before uploading, define:

• the decision or evaluation question;
• the relevant time period;
• the population and context;
• which sources are primary;
• which sources are secondary;
• which sources are authoritative;
• which sources are duplicates or versions of the same document;
• which materials are AI-generated or analyst-generated;
• which documents are outdated or superseded.

Suggested source architecture

8. Protocol 6: Compass

For each new development, ask:

1. Does it affect our current work?
2. Does it change a methodological choice?
3. Does it create a material opportunity?
4. Does it create a material risk?
5. Does it require a workflow change?
6. What can we safely ignore for now?

The purpose is not to know everything. It is to identify what is relevant enough to deserve professional attention.

9. Protocol 7: Unblock

Your first output could be:

• three emerging themes;
• a draft evidence matrix;
• five competing explanations;
• a preliminary theory-of-change gap analysis;
• a list of evidence gaps;
• a draft evaluation question hierarchy.

10. Protocol 8: The Advocate

1. State your current verdict.
2. Ask the AI to construct the strongest evidence-supported alternative interpretation.
3. Identify evidence supporting that interpretation.
4. Identify its strongest weaknesses.
5. Only then conduct the critique.

This can be especially useful for evaluation recommendations, programme design reviews, policy analysis, and contested findings.

11. Protocol 9: The Threshold

12. Protocol 10: The Ledger

Claim
  ↓
Interpretation
  ↓
Document
  ↓
Original source
  ↓
Underlying evidence

This matters when teams repeatedly upload AI-generated summaries, draft reports, or previous syntheses back into the same workspace.

13. Applying the Protocols Across the M&E Cycle

14. A Combined Workflow for Evaluation Evidence Synthesis

Suppose your team needs to answer:

1. Architect: define the question and organise the source set.
2. Radar: distinguish primary evidence, interpretation, repetition, and contradiction.
3. Deep Drill: examine the strongest findings and trace their supporting evidence.
4. Ledger: trace important claims back to original sources.
5. Advocate: construct the strongest alternative explanation.
6. Crucible: stress-test the emerging conclusion.
7. Control Problem: run decision gates before accepting the conclusion.
8. Human review: assess methodological validity, uncertainty, and implications.

Notice what is missing: a step called “Ask the AI whether the conclusion is correct.” AI can support the investigation, but it should not become the final authority on the validity of an evaluation conclusion.

15. AI Research Protocols for Qualitative Evidence

Notebook-based AI can help teams work across interview transcripts, focus group notes, case studies, and qualitative reports. The same protocols apply, but qualitative evidence requires particular care around context and interpretation.

Before analysis

• Define the research question.
• Document the sampling approach and population.
• Identify the coding framework if one exists.
• Separate participant quotations from analyst interpretations.
• Document relevant contextual information.

During analysis

• Ask AI to propose themes, not automatically declare them final.
• Trace themes back to underlying excerpts.
• Look for negative cases and disconfirming evidence.
• Preserve differences between participant groups.
• Check whether a theme is genuinely recurring or simply prominent in a small number of sources.

Before reporting

• Review the interpretation against the original excerpts.
• Check whether context has been lost.
• Check whether quotations support the interpretation being made.
• Distinguish participant perspectives from evaluator conclusions.

16. AI Research Protocols for Quantitative and Monitoring Evidence

AI can assist with quantitative analysis, but structured checks are essential. A notebook should not be allowed to infer meaning from a number without understanding the metric and the dataset that produced it.

Before interpreting a result, verify:

• indicator definition;
• numerator and denominator;
• unit of measurement;
• data grain;
• population;
• reporting period;
• missing-data treatment;
• disaggregation;
• data-quality flags;
• calculation method.

17. AI Research Protocols for Programme and Policy Documents

Development professionals often work with strategies, logframes, implementation plans, donor reports, policy documents, and programme reviews. These documents may use overlapping terminology while carrying different levels of authority.

A useful protocol asks AI to identify:

• the document’s purpose and intended audience;
• the decision or policy question addressed;
• definitions of key terms;
• stated assumptions;
• commitments versus aspirations;
• evidence-backed statements versus assertions;
• implementation responsibilities;
• monitoring and reporting requirements;
• conflicts with other authoritative documents.

18. Provenance: Make the Path Back to the Evidence Visible

Provenance means being able to understand where information came from and how it was transformed. For AI-assisted M&E, provenance is a practical quality-control mechanism.

For an important finding, record where possible:

• original source;
• source date and version;
• dataset or document identifier;
• data collection method;
• cleaning or transformation performed;
• analysis method;
• analyst or responsible team;
• approval or review status;
• report in which the finding was used.

For teams that want a formal provenance model, the W3C PROV-O recommendation provides a standard way to represent provenance relationships. Most M&E teams do not need to implement PROV-O directly; the practical lesson is to keep important results traceable to their sources.

19. FAIR Principles and AI-Ready Evidence

The FAIR principles focus on making data and metadata Findable, Accessible, Interoperable, and Reusable. For M&E teams, that translates into practical questions:

• Can another team understand an indicator without asking its creator?
• Can a dataset be connected to its metadata?
• Can an AI system identify the source of an important result?
• Are important terms defined consistently?
• Can findings be traced to evidence?
• Can another system reuse the information responsibly?

You can learn more from the GO FAIR Principles. You do not need to implement every FAIR principle before using AI. Start by improving the quality, consistency, findability, and traceability of the information your workflows depend on.

20. A Reusable AI Protocol Prompt

The following can be adapted for a notebook-based M&E research workflow:

You are supporting an M&E professional with evidence analysis.
Do not treat repeated statements as independent evidence.
Distinguish:
1. Primary evidence
2. Secondary interpretation
3. Generated or derived material
For important claims:
- identify the original source;
- identify the reporting period and population where relevant;
- identify important methodological limitations;
- distinguish evidence from interpretation.
Do not:
- treat missing information as evidence of absence;
- infer causality from descriptive evidence;
- silently resolve contradictions;
- treat AI-generated summaries as independent corroboration;
- force a conclusion when evidence is insufficient.
Before producing a final synthesis:
1. State the main evidence-supported findings.
2. Identify the evidence supporting each finding.
3. Identify important conflicting evidence.
4. Identify alternative explanations.
5. Identify what cannot be concluded.
6. Identify remaining evidence gaps.
7. State where human review is required.
If the evidence is insufficient, return:
HOLD — additional review or evidence required.
If the question cannot responsibly be answered from the supplied material, return:
BLOCKED — insufficient evidence.

21. A 30-Minute Practice Exercise

Use one real M&E or development question that your team is currently working on.

22. A Protocol Selection Guide

23. What These Protocols Do Not Solve

A good protocol cannot turn weak evidence into strong evidence.

It cannot compensate for a poorly designed evaluation, a biased sample, an invalid indicator, missing data, an inappropriate analytical method, or inadequate contextual understanding.

It can, however, make those limitations easier to see and harder to overlook.

AI research protocols also do not remove the need to consider:

• confidentiality and data protection;
• participant privacy;
• research ethics;
• bias and representation;
• methodological validity;
• reproducibility;
• human oversight;
• appropriate governance for consequential decisions.

24. The Bigger Lesson for AI-Assisted Development Work

AI readiness is often framed as a question of which model or tool an organisation should adopt.

For many M&E and development teams, a more fundamental question comes first:

If the answer is no, adding a more capable AI system may simply make the existing workflow faster.

Better AI-assisted research starts with better research architecture: clear questions, organised sources, explicit definitions, provenance, verification points, and human judgement.

25. Practical Next Step

Pick one recurring workflow this week: an evaluation synthesis, donor report, literature review, data-quality investigation, programme review, policy analysis, or learning exercise.

Do not start by asking the AI for the final answer. Start with:

1. Architect: What information should be in the workspace?
2. Radar: What is evidence, interpretation, repetition, and contradiction?
3. Deep Drill: What supports the most important claim?
4. Advocate: What is the strongest alternative explanation?
5. Crucible: What could make this conclusion fail?
6. Control Problem: Is the work ready to proceed?

Further EvalCommunity Resources

Continue building practical AI and M&E capability with these resources:

Explore the AI Toolkit for Evaluators

Browse the M&E Framework Library

Review the Principles for AI Use in M&E

Assess an AI Tool for M&E Use

Explore the AI for M&E Professional Bundle

EvalCommunity Academy · Practical AI workflows for monitoring, evaluation, learning, evidence, and international development.