Can AI Be Accurate and Still Mislead an Evaluator
Can AI Be Accurate and Still Mislead an Evaluator? An M&E Experiment
An AI-generated evaluation finding can be factually accurate and still give an evaluator the wrong impression. This practical experiment shows how.
The issue is not that accuracy is unimportant. Accuracy matters. The problem is assuming that accuracy alone tells us whether an AI-generated evaluation finding is trustworthy.
This article builds on the
Accuracy Paradox of AI
and the previous tutorial,
Verification of an AI-Generated Evaluation Finding
.
The underlying Accuracy Paradox paper argues that an overreliance on accuracy can leave deeper problems involving epistemic reliability, user reliance, interpretation and transparency insufficiently addressed. :contentReference[oaicite:2]{index=2}
The Experiment
We will use a simple M&E scenario and compare several versions of an AI-generated finding based on the same underlying evidence.
The purpose is not to determine which AI model is best. It is to demonstrate how the same evidence can produce different interpretations depending on what the AI is asked to include, exclude or examine.
The Evaluation Scenario
Imagine an employment programme for young people.
An evaluation collects information from programme participants and a comparison group.
The underlying evidence shows:
- Employment increased among programme participants.
- The increase was larger among participants who completed the full programme.
- Employment also increased among the comparison group.
- Results varied between regions.
- Some participant groups experienced little or no improvement.
- The evidence does not establish that every observed change was caused by the programme.
Now ask AI to summarize the evaluation finding.
Version 1 — The Short AI Finding
AI-generated finding:
“The programme improved employment outcomes among young people.”
The statement sounds reasonable. But several important elements have disappeared:
- The comparison group.
- Regional differences.
- Differences between participant groups.
- The distinction between association and causation.
- The fact that not everyone improved.
The statement may be directionally consistent with the evidence, but it does not communicate the full evidentiary picture.
Version 2 — Add More Precision
Now ask AI to provide a more precise finding.
AI-generated finding:
“Employment increased among young people who participated in the programme, with larger gains among those who completed the full programme.”
This is more informative, but important questions remain:
- Did employment also increase among non-participants?
- Were results consistent across regions?
- Did all participant groups experience similar outcomes?
- Can the increase be attributed to the programme?
Version 3 — Add the Comparison Group
Now explicitly ask AI to preserve the comparison group and identify uncertainty.
Prompt:
“Summarize the employment finding and explicitly compare the results with the comparison group. Identify important uncertainty.”
Possible AI-generated finding:
“Employment increased among programme participants, particularly among those who completed the programme. However, employment also increased among the comparison group, so the available evidence should be interpreted cautiously when attributing the change to the programme.”
The underlying evidence did not change.
The output changed because the prompt required the AI to preserve more of the evidence.
Version 4 — Search for Contradictory Evidence
Now make the test more demanding.
Prompt:
“Identify evidence in the evaluation that could contradict, qualify or weaken the main finding. Do not only summarize evidence supporting the programme.”
The analysis may identify:
- Employment increased in both groups.
- Results differed across regions.
- Some groups showed little improvement.
- Completion was associated with stronger outcomes, but completion may also be related to participant characteristics.
The original statement now requires substantially more qualification.
What Just Happened?
The AI did not necessarily make a simple factual error.
Instead, the problem involved selection, omission and interpretation.
This distinction is central to the Accuracy Paradox: an output does not have to be plainly false to create epistemic problems or encourage excessive reliance. :contentReference[oaicite:3]{index=3}
Accuracy Test vs. Trustworthiness Test
| Accuracy | Broader verification |
|---|---|
| Is the statement factually supported? | Does it adequately represent the evidence? |
| Is the number correct? | Is the necessary context included? |
| Does the source exist? | Does the source justify the conclusion? |
| Is the wording technically correct? | Are uncertainty and contradictory evidence visible? |
| Is the result reproduced? | Could the wording cause an evaluator to overestimate the evidence? |
This is why the EvalCommunity verification workflow emphasizes not only accuracy, but also coverage, nuance, dissent, traceability and human review. :contentReference[oaicite:4]{index=4}
The Most Important Result
This means prompt design becomes part of the evaluation workflow.
Compare:
with:
The second prompt creates a much stronger opportunity for critical review.
A Better M&E Prompt
Prompt template:
“Analyze the evidence and provide:
1. The main finding;
2. The evidence supporting it;
3. Evidence that contradicts or qualifies it;
4. Important contextual differences;
5. Subgroups with different results;
6. Alternative explanations;
7. What cannot be concluded from the evidence;
8. The appropriate level of certainty;
9. Claims requiring human verification.
Do not strengthen the conclusion beyond what the evidence supports.”
This builds directly on the
10-step verification process
from the previous article.
Five Things to Test
- Accuracy: Is the factual content supported?
- Completeness: Has important evidence been omitted?
- Context: Are relevant population, location, time and implementation differences visible?
- Uncertainty: Does the wording reflect the strength of the evidence?
- Interpretation: Has AI moved beyond what the evidence can establish?
These checks complement the
AI Tool Evaluation Protocol for M&E
,
which emphasizes defining what was tested, how it was tested, documenting errors and understanding why the result should be trusted. :contentReference[oaicite:5]{index=5}
Why This Matters for M&E
Evaluation is not simply about producing statements that are factually correct. It is about constructing defensible judgments from evidence.
That requires attention to:
- what the evidence says;
- what the evidence does not say;
- how strong the evidence is;
- which groups experience different outcomes;
- what alternative explanations exist;
- and how much uncertainty remains.
The EvalCommunity guide
Use of AI in Evaluation
similarly emphasizes human review, evidence traceability, bias checks, validation and professional evaluation judgment. :contentReference[oaicite:6]{index=6}
The Evaluator’s Experiment
You can reproduce this basic experiment using your own evaluation material.
- Choose one completed evaluation report.
- Select one finding supported by multiple pieces of evidence.
- Ask AI for a short summary.
- Ask AI for a detailed evidence-based summary.
- Ask AI to identify contradictory evidence.
- Ask AI to identify missing context.
- Ask AI to identify alternative explanations.
- Compare every output with the original evidence.
- Record what was omitted, changed or overstated.
- Have a human evaluator determine which version is appropriate.
The
AI-Assisted Consultation Documentation
tutorial provides a closely related evidence chain:
Original evidence → AI-assisted synthesis → Human verification → Interpretation → Evaluation finding. :contentReference[oaicite:7]{index=7}
What Should You Record?
| Test | Record |
|---|---|
| Accuracy | Errors, unsupported numbers or incorrect statements. |
| Coverage | Important supporting or contradictory evidence omitted. |
| Context | Relevant population, location or implementation differences omitted. |
| Uncertainty | Overconfident or appropriately qualified language. |
| Interpretation | Claims that go beyond the evidence. |
The Main Lesson
An AI system can accurately reproduce a number while omitting the comparison group.
It can accurately describe an increase while failing to mention that the increase occurred in both the intervention and comparison groups.
It can accurately summarize positive findings while failing to surface contradictory evidence.
And it can use confident language that makes an association sound causal.
These problems can arise through selection, framing, omission and interpretation, even without an obvious fabricated fact.
Related EvalCommunity Resources
- Accuracy Paradox of AI — why apparent AI accuracy is not the whole story.
- Verification of an AI-Generated Evaluation Finding — the 10-step verification process.
- Avoid Fake Citations When Using AI — verify sources, quotations, statistics and evidence claims. :contentReference[oaicite:8]{index=8}
- How GenAI Works for Evaluators — understand hallucination, traceability and evidence grounding. :contentReference[oaicite:9]{index=9}
- AI Tool Evaluation Protocol for M&E — systematically test AI tools and document errors. :contentReference[oaicite:10]{index=10}
- AI-Assisted Consultation Documentation — test accuracy, coverage, nuance, dissent and traceability. :contentReference[oaicite:11]{index=11}
- AI Prompt That Finds Its Own Mistakes — use structured self-auditing before using AI outputs. :contentReference[oaicite:12]{index=12}
Final Takeaway
The question is not simply whether AI can be accurate.
The more important question for evaluators is:
Accurate enough for what decision, based on what evidence, with what context, and with what uncertainty?
That is the difference between checking an AI answer and evaluating an AI-assisted evidence process.
For M&E professionals, a stronger workflow is:
AI → Finding → Evidence Check → Context Check → Contradiction Check → Human Judgment → Report
The goal is not to make AI sound less confident.
The goal is to make evaluation conclusions more defensible.
Authoritative Sources
- NIST — Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile
- NIST — ARIA Evaluation Planning Manual
- NIST — AI Measurement and Evaluation
- WHO — Safe and Ethical AI
This article presents a practical EvalCommunity experiment illustrating how AI-generated evaluation findings can change when evidence, context, contradictory information and uncertainty are explicitly tested. It is not presented as a controlled scientific benchmark of a particular AI model.
