Back
Can AI Turn Correlation Into Causation? A Practical M&E Test

Can AI Turn Correlation Into Causation?

Can AI Turn Correlation Into Causation? A Practical M&E Test

A practical tutorial for detecting when an AI-generated evaluation finding goes beyond what the evidence actually supports.

AI can identify patterns, compare groups, summarize results and draft evaluation findings in seconds.

But there is a critical distinction between saying that two things are associated and saying that one caused the other.

For Monitoring and Evaluation professionals, this distinction is fundamental. An outcome can improve after a programme without the programme necessarily being the reason for that improvement.

This tutorial presents a simple experiment that M&E professionals can use to test an AI-generated finding for causal overreach before using it in an evaluation report.

1. Why Causality Matters in Evaluation

Impact evaluation is concerned with whether an intervention caused a change, not simply whether a change occurred.

UNDP’s 2026 Impact Evaluation Guidelines emphasize that impact evaluations test whether a programme caused the change observed and explain the importance of credible counterfactuals and appropriate evaluation methods.
UNDP Impact Evaluation Guidelines.

The World Bank’s Impact Evaluation in Practice also treats causal inference and counterfactuals as central to impact evaluation and covers methods such as randomized assignment, instrumental variables, regression discontinuity, difference-in-differences and matching.
World Bank: Impact Evaluation in Practice.

This creates an important risk when AI is used to interpret evaluation evidence:

An observed change can be turned into a causal-sounding finding without sufficient evidence for causal attribution.

2. A Simple M&E Experiment

Consider this illustrative evaluation scenario.

Youth Employment Programme
Participants: 800 young people
Employment before programme: 48%
Employment after programme: 67%
Comparison group: 52% → 61%

The participant group increased by 19 percentage points.

The comparison group increased by 9 percentage points.

There is therefore a difference in the amount of change between the two groups.

But the numbers alone do not tell us whether the programme caused the entire difference.

The interpretation depends on the evaluation design, how the groups were constructed, timing, assumptions, other influences and the method used to estimate the causal effect.

3. What Might AI Say?

Give the evidence to an AI system and use a simple prompt:

Prompt

“Review these evaluation results and write a concise finding explaining the effect of the youth employment programme.”

A possible AI-generated response could be:

“The youth employment programme increased employment among participants from 48% to 67%, demonstrating a positive impact of the intervention.”

The first part describes an observed change.

The phrase “demonstrating a positive impact” makes a much stronger claim.

The important question is not whether the sentence sounds reasonable. It is:

What evidence supports the causal interpretation?

4. Separate Three Different Claims

ClaimWhat it means
Outcome changedThe measured outcome was different over time or between groups.
Association existsTwo variables, groups or exposures are related in the observed data.
Causal effect existsThe evaluation provides evidence that the intervention caused the observed effect.

These are not interchangeable statements.

A change can be observed without establishing causality. An association can exist without establishing causality. A causal claim requires an appropriate evaluation design, evidence and assumptions that support the causal interpretation.

5. Ask AI to Stress-Test Its Own Causal Claim

Now repeat the experiment with a more demanding prompt:

Prompt for evaluators

“Review the evidence as a critical evaluation reviewer.

Separate:

• observed changes;
• associations;
• causal claims;
• evidence supporting each causal claim;
• alternative explanations;
• possible confounding factors;
• assumptions required for the interpretation;
• limitations of the evaluation design;
• what cannot be concluded from the evidence.

Do not convert correlation, association or temporal sequence into causation. For every causal statement, identify the specific evidence and evaluation design supporting it.”

This changes the AI’s task from writing a finding to testing the finding.

That is the key practical shift.

6. Look for Causal Language

A quick quality-assurance technique is to search the AI output for causal language.

  • caused
  • led to
  • resulted in
  • produced
  • demonstrated impact
  • because of the programme
  • as a result of the intervention

These words are not automatically wrong.

They simply signal that the evaluator should ask whether the evidence and evaluation design justify the causal interpretation.

For every causal statement, ask: “What evidence supports this causal claim?”

7. The Counterfactual Question

For impact questions, one of the most important questions is:

What would have happened without the intervention?

That is the counterfactual question.

UNDP’s 2026 guidance explains that counterfactual analysis helps evaluators distinguish the effects of an intervention from other external influences.
Read the UNDP guidance.

The World Bank similarly explains that simply observing an outcome improve after participation is not sufficient to establish causality, because the outcome might have changed even without the intervention.
See Impact Evaluation in Practice.

AI can help an evaluator identify possible counterfactual questions, assumptions and alternative explanations.

But an AI-generated answer does not itself create a credible counterfactual.

8. Five Questions to Test an AI Causal Claim

  1. What exactly changed? Identify the outcome and the observed change.
  2. Compared with what? Identify the baseline, comparison or counterfactual basis.
  3. What else could explain the change? Identify plausible competing explanations.
  4. Does the evaluation design support causal attribution? Check the actual methodology and assumptions.
  5. What is the strongest defensible conclusion? Do not use stronger causal language simply because AI produces it fluently.

9. A Reproducible M&E Workflow

  1. Select an evaluation finding involving an observed change.
  2. Give the underlying evidence to an AI system.
  3. Ask AI to write a conventional finding.
  4. Save the output before editing it.
  5. Highlight every causal statement.
  6. Ask AI to identify the evidence supporting each causal statement.
  7. Ask AI to identify alternative explanations.
  8. Check the actual evaluation design and assumptions.
  9. Rewrite the finding using only defensible claims.
  10. Complete human methodological review.

10. Add a “Causality Check” to Your AI Workflow

A simple additional quality-control step can be added to an AI-assisted evaluation workflow:

AI Analysis → Observed Change → Association Check → Causal Evidence Check → Human Interpretation

The purpose is not to make AI less useful.

The purpose is to prevent a fluent explanation from becoming a causal conclusion without an appropriate evidence check.

11. What AI Can Help With

  • Identifying relationships and patterns.
  • Generating alternative explanations.
  • Identifying possible confounders for human review.
  • Explaining causal-inference concepts.
  • Stress-testing a draft finding.
  • Suggesting additional questions for analysis.
  • Structuring evidence and assumptions for review.

These uses position AI as an analytical assistant rather than as the final methodological decision-maker.

12. What Still Requires Human Judgment

  • Whether the evaluation design can support causal attribution.
  • Whether the assumptions required by the method are credible.
  • Whether the comparison or counterfactual is appropriate.
  • Whether alternative explanations have been adequately addressed.
  • Whether the final causal wording is justified.
  • The final evaluation conclusion.

13. Why Accuracy Alone Is Not Enough

There is a direct connection between this problem and the first tutorial in this series:
Accuracy Paradox of AI: What It Means and How to Fix It.

An AI-generated statement can contain accurate numbers and still lead the reader toward an interpretation that the evidence does not adequately support.

The 2026 research paper Accuracy Paradox: Addressing Epistemic, Manipulative, and Societal Risks of Hallucination in AI Governance argues that narrow accuracy should not be treated as equivalent to broader trustworthiness and highlights the importance of context, uncertainty and epistemic reliability.

For evaluation, this means that checking whether AI repeated the numbers correctly is only one part of quality assurance.

Source:
Accuracy Paradox — Computer Law & Security Review, 2026.

14. Connect This Tutorial With the Previous Series

This tutorial is the sixth part of the EvalCommunity AI + M&E evidence-quality series.

15. The Series Is Building an Evidence-Quality Workflow

Taken together, the six tutorials create a progressively stronger quality-assurance process:

Accuracy
→
Verification
→
Context
→
Completeness
→
Disagreement
→
Causality
→
Human Judgment

The objective is not to prevent evaluators from using AI.

It is to prevent increasingly fluent AI systems from quietly becoming substitutes for evidence review and professional evaluation judgment.

16. Related EvalCommunity Resources

17. Authoritative Sources

18. The Practical Rule

Do not let AI turn “what changed” into “what caused the change” without checking the causal evidence.

AI can help evaluators explore relationships, formulate hypotheses, identify alternative explanations and stress-test interpretations.

But a polished AI explanation is not a substitute for an appropriate evaluation design.

The evaluator still needs to determine whether the evidence supports the causal claim.

Note: The numerical example in this tutorial is illustrative. It is designed to demonstrate a quality-assurance method and is not an empirical evaluation result.

Academy

The courses and articles are developed by a team of experienced evaluators, collaborators, authors, and software developers, guided by Fation Luli. EvalCommunity Academy combines practical expertise in Monitoring & Evaluation and International Development with the latest advances in AI to create high-quality, accessible, and practical learning experiences for professionals worldwide.