EvalCommunity Academy · Case Study

AI Anchoring Bias in M&E and Development Decision-Making

How an AI recommendation can become the starting point for human judgement — and how evaluators and development professionals can deliberately reopen the frame.

What you will learn

This case study explores how AI-generated recommendations can influence human judgement, why the first number or interpretation presented can matter, and how the Consider-the-Opposite (CTO) approach can help create a deliberate pause before accepting an AI-supported conclusion.

The goal is not simply to understand the research. It is to translate the underlying lesson into practical habits that can be used in evaluation, monitoring, programme management, strategy, reporting, research and international development.

1. The situation

Imagine that an evaluator is reviewing evidence about programme performance and asks an AI system to help interpret the available information.

The AI produces an initial assessment:

“The programme is likely to achieve 92% of its annual target.”

The evaluator now has a starting point.

Even if the evaluator knows that the AI can be wrong, the recommendation may influence the subsequent analysis. The evaluator may begin asking whether the programme can reach 92%, rather than first asking what the evidence independently indicates.

This is the central issue explored in the research examined in this case study: can an AI recommendation become an anchor for human judgement?

2. What is anchoring bias?

Anchoring bias occurs when an initial piece of information influences subsequent judgement, even when that initial information may not be sufficiently relevant or reliable.

In an AI-supported workflow, the anchor could be:

a numerical estimate;
a predicted outcome;
a classification;
an interpretation of qualitative evidence;
a suggested recommendation;
or an initial explanation generated by an AI system.

The important point is that the influence of the anchor does not necessarily disappear simply because a human knows that the recommendation came from AI.

3. What did the research examine?

The case study is based on research by Lemuria Carter and Dapeng Liu, published in the International Journal of Information Management in 2025.

The study, “How was my performance? Exploring the role of anchoring bias in AI-assisted decision making,” examined whether AI-generated recommendations could influence human decisions through anchoring.

The original research concerned employee performance appraisal. It was not an M&E study.

Research boundary

The findings described below come from research on employee performance appraisal. The M&E and international development examples in this case study are educational applications developed by EvalCommunity Academy. They should not be interpreted as empirical findings from the original research.

4. What did the experiments find?

In the first study, participants evaluated employee performance under different conditions.

The researchers varied two factors:

Recommendation source: AI or human
Anchor: a high or low initial performance score

The high anchor was 91/100, while the low anchor was 51/100.

For the AI condition, participants exposed to the high anchor gave an average performance rating of 89.64, compared with 72.66 under the low-anchor condition.

The researchers reported a statistically significant difference between the two conditions, with a very large effect.

The non-AI conditions showed a similar directional pattern: the high-anchor group had an average rating of 92.44, while the low-anchor group averaged 71.27.

The study therefore provides evidence that the initial recommendation or value presented to participants could influence subsequent judgement.

5. The important lesson for evaluators

The lesson is not that AI is biased or that AI recommendations should not be used.

The more useful lesson is this:

An AI-generated recommendation can become part of the reasoning environment in which a human makes a decision.

Once an initial assessment is visible, it can influence what questions are asked, which evidence receives attention, what alternatives are considered and how subsequent information is interpreted.

This matters particularly in M&E and development because many professional judgements involve incomplete evidence, competing explanations, contextual information and uncertainty.

6. When a working draft becomes the frame

One of the most important implications goes beyond the individual evaluator.

Consider a team using AI to prepare an initial synthesis of programme evidence.

The AI produces a working draft. The team then reviews it, corrects several statements and adds additional evidence.

At first glance, this appears to be a normal quality-assurance process.

But there is another possibility.

The original AI draft may already have influenced the team’s frame for understanding the problem.

It may influence:

which issues are discussed;
which explanations appear plausible;
which questions are considered important;
where disagreement occurs;
which options appear realistic;
and eventually which conclusions are considered reasonable.

This creates an important distinction:

Reviewing an AI-generated draft is not necessarily the same thing as questioning the frame created by that draft.

7. From individual judgement to collective judgement

AI-assisted synthesis can increasingly become a shared object inside a team.

An evaluator may generate an initial analysis. Another team member reviews it. A programme manager comments on it. A technical adviser adds evidence. The group eventually develops a common interpretation.

The final product may therefore be human-reviewed, but the initial AI-generated framing may still have influenced the path the group followed.

This is why AI provenance can become important.

Teams should be able to identify where AI was used, what it contributed, what evidence was independently verified, what human judgement changed the output and where uncertainty remains.

8. The Consider-the-Opposite strategy

The researchers also examined a debiasing intervention known as Consider-the-Opposite (CTO).

The basic idea is straightforward: before accepting an initial judgement, deliberately consider why the opposite conclusion might be true.

In the research, the CTO intervention reduced the influence of the high anchor.

This provides a useful practical principle for AI-assisted work:

Do not only ask:

“Does this AI recommendation make sense?”

Also ask:

“What evidence or reasoning would make this recommendation wrong?”

9. Applying CTO to M&E

Suppose an AI system concludes that a programme is performing well because the majority of reported indicators are on track.

An evaluator could accept the conclusion and begin drafting the performance section of the report.

Instead, apply CTO.

Ask:

What if the programme is not performing as well as the AI suggests?
What evidence could contradict the conclusion?
Could the indicators be masking implementation problems?
Could data-quality problems be inflating reported performance?
Could contextual changes explain the observed results?
Could the theory of change be incomplete?

The purpose is not to automatically reject the AI recommendation.

The purpose is to reopen the reasoning process before the recommendation becomes the conclusion.

10. What evaluators and development professionals can learn

This case study should not end with “understand anchoring bias.” The more valuable outcome is to develop a practical professional capability.

After studying this case, professionals should be able to:

recognize when an AI output may become an anchor;
separate an AI recommendation from independently verified evidence;
challenge an initial AI interpretation;
use structured counter-analysis before accepting an AI-supported conclusion;
document where AI influenced an analytical process;
reopen the analytical frame when working with teams;
and retain human accountability for professional judgement.

11. A practical learning exercise

The following exercise can be used individually or with an M&E/development team.

Step 1 — Choose a real analytical task.
Use a task such as interpreting monitoring data, reviewing qualitative findings, assessing an indicator, developing an evaluation question or synthesizing programme evidence.

“`

Step 2 — Record the first AI recommendation.
Do not immediately improve or rewrite it. Capture what the AI initially says.

Step 3 — Identify the potential anchor.
What number, conclusion, classification or interpretation could influence your subsequent thinking?

Step 4 — Apply Consider-the-Opposite.
Ask what evidence would support the opposite conclusion.

Step 5 — Return to the evidence.
Review the original data, sources, definitions, assumptions, context and data-quality information.

Step 6 — Make your independent assessment.
Write your judgement before comparing it with the AI recommendation again.

Step 7 — Compare the two positions.
Identify where the AI and your independent assessment agree and where they differ.

Step 8 — Reopen the frame with the team.
If the work is collaborative, ask whether the initial AI output influenced the questions, assumptions or options considered by the group.

12. Where this can be applied

Evaluation design: Challenge AI-generated evaluation questions, assumptions, criteria and proposed methods before adopting them.

Theory of Change: Ask what alternative causal pathways or explanations could exist beyond the AI-generated theory.

Qualitative analysis: Check whether an AI-generated theme or interpretation is causing the analyst to overlook contradictory evidence.

Data quality: Challenge conclusions based on apparently strong indicators by examining definitions, sources, missing data, collection processes and verification.

Evaluation reporting: Identify whether an AI-generated narrative has unintentionally established the direction of the final report.

Programme strategy: Challenge AI-generated recommendations by deliberately considering alternative strategic options and their supporting evidence.

13. Questions that keep the frame open

When AI becomes part of everyday analytical work, professionals can build a habit of asking:

What did the AI assume?
What did the AI leave out?
What evidence contradicts this interpretation?
What would we conclude without seeing the AI’s recommendation?
Are we evaluating the evidence, or evaluating the AI’s interpretation of the evidence?
Has the initial AI output influenced the questions we are asking?
What alternative explanation deserves consideration?

14. The practical lesson

AI should not simply produce an answer. It should become part of a reasoning process that remains open to challenge.

The professional responsibility is not to eliminate AI from decision-making. It is to prevent an initial AI recommendation from quietly becoming the unquestioned frame for subsequent judgement.

15. Conclusion

AI can significantly expand the capacity of evaluators and development professionals to analyze information, synthesize evidence and support decisions.

But greater analytical capacity also creates a new professional responsibility: knowing when to challenge the machine’s starting point.

The research examined in this case study provides evidence that an initial recommendation can influence subsequent human judgement. The Consider-the-Opposite approach offers one practical mechanism for reducing that influence.

For M&E and international development professionals, the broader lesson is to develop workflows in which AI recommendations are treated as inputs to professional reasoning rather than conclusions that determine it.

Disclaimer

This case study is based on the research article by Carter and Liu (2025), which investigates anchoring bias in AI-assisted employee performance appraisal.

The original research does not study Monitoring & Evaluation, international development, humanitarian programmes or development-sector decision-making.

Examples involving M&E and international development are educational adaptations created by EvalCommunity Academy to help professionals understand how the research concept may be relevant to their own work. They should not be interpreted as findings from the original study.

Original research

Carter, L. & Liu, D. (2025). How was my performance? Exploring the role of anchoring bias in AI-assisted decision making. International Journal of Information Management, 82, 102875.

View the original research article

Continue learning with EvalCommunity Academy

Explore practical resources for using AI responsibly and effectively in evaluation and international development:

EvalCommunity Academy · Practical learning for evaluators and international development professionals working in an AI-enabled world.

AI Anchoring Bias in M&E Decision-Making