How to Explain AI to Skeptical M&E Colleagues

AI is becoming increasingly common in Monitoring and Evaluation, but not everyone is convinced that it belongs in evaluation work.

That skepticism is understandable.

Experienced evaluators have spent years developing skills in research design, data analysis, qualitative coding, interpretation, and critical thinking. They know that evidence needs context, that data can be misleading, and that professional judgment matters.

So when someone says, “AI can analyze your evaluation faster,” a natural response may be:

“But can I trust it?”

That is the right question.

The goal is not to convince colleagues that AI is perfect. Instead, explain what AI can do, where it can help, where it can fail, and why the evaluator must remain responsible for the final judgment.

AI as a Research Assistant

Imagine hiring a research assistant who can:

• Read hundreds of documents very quickly
• Extract information from reports
• Organize qualitative responses
• Identify recurring themes
• Summarize interviews
• Compare documents
• Draft tables and reports
• Suggest possible patterns in data
• Help formulate questions

That sounds useful. But there is a catch.

The research assistant may misunderstand something, overlook an important contextual detail, interpret a statement incorrectly, or confidently provide an answer that is wrong.

You would not allow an inexperienced research assistant to publish an evaluation report without reviewing their work.

AI should be treated in much the same way.

The analogy: AI is like a very fast research assistant. It can help you process information and generate ideas, but you remain the expert responsible for deciding what is correct, relevant, ethical, and meaningful.

What AI Can Do for an Evaluator

Consider a typical evaluation project. You may have a theory of change, evaluation questions, interview transcripts, survey data, focus group notes, program documents, monitoring data, and previous evaluation reports.

AI can assist with many parts of this workflow, particularly where large amounts of information or repetitive tasks are involved.

Before Data Collection

• Review an evaluation framework
• Identify potential gaps in evaluation questions
• Generate draft interview questions
• Suggest indicators
• Develop preliminary coding frameworks
• Examine a theory of change
• Draft survey questions
• Identify potential risks or assumptions

During Data Collection

• Transcription
• Translation
• Interview summarization
• Organization of field notes
• Identification of recurring topics
• Preparation of follow-up questions

During Analysis

• Clean and structure text
• Classify qualitative responses
• Apply a preliminary codebook
• Identify potential themes
• Compare stakeholder groups
• Summarize large document collections
• Identify possible contradictions
• Generate descriptive summaries

During Reporting

• Draft sections of reports
• Summarize findings
• Create executive summaries
• Develop presentation content
• Draft recommendations
• Adapt technical findings for different audiences

Important: AI assistance is not the same as AI judgment. The evaluator still needs to determine whether the analysis is valid.

AI Does Not Replace Evaluation Expertise

An AI system may identify that a particular theme appears frequently in interview transcripts.

But frequency does not necessarily mean importance.

For example, suppose 70% of participants mention transportation difficulties. That may be an important finding. But an evaluator may discover that the remaining 30% include particularly vulnerable participants who face a completely different barrier.

Context changes interpretation.

Similarly, AI may identify a positive trend in program data. An experienced evaluator might ask:

• Was the measurement method changed?
• Did the sample change?
• Were some participants excluded?
• Is there a selection effect?
• Did external circumstances influence the result?
• Is the apparent improvement actually meaningful?
• Is the indicator measuring what we think it is measuring?

AI can help identify patterns. Evaluation expertise determines what those patterns mean.

A 30-Second Explanation of AI for an M&E Colleague

Think of AI as a very fast research assistant. It can read large amounts of information, summarize documents, organize qualitative data, identify possible patterns, and help draft analysis. But like a research assistant, it can make mistakes and does not automatically understand the context of a program. We still need to check its work and apply our professional judgment. The value is not replacing the evaluator. It is reducing the time spent on repetitive work so we can spend more time on interpretation, critical thinking, and decision-making.

Explaining AI to a Program Manager

AI can help an M&E team process information faster and make routine analytical tasks more efficient. For example, it can help summarize interviews, organize qualitative responses, compare reports, identify recurring themes, and prepare first drafts of analysis. However, AI does not remove the need for professional review. The M&E team remains responsible for validating findings, interpreting results in context, protecting sensitive information, and making decisions based on appropriate evidence.

Three Questions Skeptical Evaluators May Ask

1. “What if AI gives me the wrong answer?”

It can. AI outputs should therefore be treated as draft analysis rather than automatically validated evidence.

AI generates → Evaluator checks → Evidence is verified → Evaluator interprets → Final conclusion

For important findings, go back to the underlying data. If AI says that a particular theme appears across 40 interviews, ask it to identify the evidence supporting the conclusion and verify that evidence yourself.

2. “Isn’t AI going to replace evaluators?”

A better question is: Which parts of evaluation are most likely to be automated, and which require human judgment?

Routine activities such as transcription, document summarization, initial categorization, data organization, drafting, and repetitive reporting are more suitable for AI assistance.

Other activities remain heavily dependent on human expertise, including understanding context, assessing validity, interpreting stakeholder perspectives, identifying ethical issues, understanding power relationships, challenging assumptions, and making judgments about evidence.

The likely future is therefore not simply “AI versus evaluators.”

It is more likely to be evaluators using AI versus evaluators working without AI.

3. “Can we trust AI with evaluation data?”

This is a serious question. The answer depends on what information is being shared, which AI system is being used, how the system handles submitted data, organizational policies, contractual requirements, applicable data protection requirements, and whether individuals can be identified.

Do not put confidential or personally identifiable information into an AI system unless your organization has determined that the system and workflow are appropriate for that data.

Where possible, remove names and identifying information and use approved organizational tools and processes.

The Best Demonstration: Use Their Own Evaluation Document

If you want to demonstrate AI’s value to a skeptical colleague, don’t start with a presentation about AI.

Start with their work.

Take a document they already know, such as:

• An evaluation report
• Interview transcripts
• A theory of change
• A monitoring report
• A survey dataset
• Focus group notes

Then choose one narrow task.

“Let’s see whether AI can identify the main themes in these interview notes.”

Ask AI to summarize the material, produce a preliminary thematic analysis, and identify supporting evidence for each theme. Then have the evaluator review the results.

Instead of asking, “Do you believe AI is useful?”, ask:

“Did this save you time, and was the output useful after you reviewed it?”

That is a much more productive conversation.

A Practical AI Demonstration Prompt

Suppose an evaluator has a collection of interview notes. Try a prompt like this:

Review these interview notes and identify the major themes.
For each theme, provide:
1. A short description
2. The evidence supporting the theme
3. Any contradictory evidence
4. Which stakeholder groups mentioned it
5. Your confidence in the finding
Do not invent information. If the evidence is insufficient, say so.
Then identify three conclusions that require particular caution and explain why an evaluator should verify them before using them in an evaluation report.

The final part of the prompt is important. You are not simply asking AI to produce an answer. You are asking it to help identify where human judgment is needed.

From “AI Knows” to “AI Assists”

One common mistake when discussing AI is saying:

“AI knows…”

A better vocabulary is:

• AI identified
• AI suggested
• AI summarized
• AI classified
• AI detected a possible pattern
• AI generated a draft
• AI provided a preliminary interpretation

This language reinforces the correct relationship between the technology and the evaluator.

AI and the M&E Principle of Triangulation

There is an interesting connection between AI and a familiar M&E principle: triangulation.

Evaluators rarely rely on a single source of evidence when an important conclusion is at stake. They may compare survey results, interviews, administrative data, observations, program documentation, and stakeholder perspectives.

The same principle can be applied to AI.

AI identifies a theme → Evaluator checks the original data → Evidence is compared → Context is considered → Evaluator determines whether the finding is credible

This is much closer to good evaluation practice than simply accepting whatever an AI system produces.

A Human-in-the-Loop Model for M&E

1. Human defines the question
   What are we trying to understand?
2. AI assists with processing
   Can AI help organize, summarize, classify, or analyze the information?
3. Human validates
   Does the output accurately represent the evidence?
4. AI helps explore
   Are there additional patterns, contradictions, or questions worth investigating?
5. Human interprets
   What does the evidence mean in this program and context?
6. Human decides
   What conclusions and recommendations are justified?

This is the human-in-the-loop approach: AI accelerates parts of the workflow while professional accountability remains with the evaluator.

The Bottom Line

You don’t need to convince skeptical colleagues that AI is perfect.

You shouldn’t.

A more credible message is that AI has significant potential, but it also has significant limitations.

It can process information quickly. It can help with repetitive analytical tasks. It can help evaluators explore large volumes of data. It can generate useful drafts and suggestions.

But it can also make errors, misunderstand context, reproduce bias, and produce convincing but unsupported statements.

That is precisely why experienced M&E professionals remain important.

The objective is not to replace evaluation expertise. The objective is to combine evaluation expertise with AI capabilities to produce better, faster, and more responsible work.

Try It Yourself

Take one evaluation document you already work with and give an AI system a small, clearly defined task.

Start with something low-risk:

• Summarize a section
• Extract key themes
• Compare two versions of a document
• Suggest interview questions
• Organize qualitative responses
• Identify potential gaps in an evaluation framework

Then check the output against the original evidence.

Don’t ask only whether AI is “right.”

Ask instead: “Did it help me do this task more efficiently without compromising the quality of my evaluation?”

This article is part of the AI in Monitoring & Evaluation learning resources from EvalCommunity Academy.