Core lesson: Evaluating an AI output is not the same as evaluating the AI-supported process.
In M&E, we also need to examine what information is retained, changed or omitted, how humans review it,
and how the resulting synthesis influences professional judgement.

The documented research

This case study is based on a documented study by Kocaballi and colleagues, published in the
Journal of the American Medical Informatics Association in 2020.

The researchers explored how an artificial intelligence documentation assistant could potentially
support future primary-care consultations. The study involved 16 practicing general practitioners
who participated in three co-design workshops.

The research is particularly relevant to M&E because evaluation teams face a similar challenge:
transforming complex human information into structured evidence without losing context, nuance or professional judgement.

Read the original study →

1. What was the study about?

The researchers explored how AI could assist healthcare professionals with documentation during
primary-care consultations.

Rather than imagining AI as an autonomous replacement for the doctor, the study explored a
human–AI collaboration in which AI could support documentation while the professional
remained involved in reviewing and approving the resulting record.

The workshops used participatory methods including affinity diagramming, brainwriting,
discussions of future solutions and video prototyping. The researchers analysed the workshop
discussions using inductive thematic analysis.

2. How could the AI assistant work?

The prototype demonstrated a possible workflow in which AI could capture consultation information,
generate documentation suggestions and allow the professional to review and modify those suggestions.

Consultation → AI-supported documentation → Human review → Final professional record

The envisioned system included capabilities such as:

• Displaying the consultation transcript.
• Highlighting potentially important information.
• Generating documentation suggestions.
• Allowing the professional to accept, reject or edit suggestions.
• Allowing the professional to inspect the surrounding source material.
• Allowing human approval of the final documentation.

This is an important distinction: the proposed workflow did not simply treat the AI-generated
output as the final professional record.

3. What did the researchers find?

Professional autonomy

Participants emphasized that technology should support professional practice rather than determine
how professionals work.

Human–AI collaboration

The study explored AI as part of a collaborative relationship with professionals rather than
simply as an autonomous replacement for them.

A wider sociotechnical system

AI documentation was considered within a wider system involving people, technology,
professional practices and organizational context.

4. Why does this matter for Monitoring & Evaluation?

M&E professionals increasingly work with large volumes of qualitative information:
interviews, focus groups, open-ended survey responses, consultation notes and stakeholder feedback.

AI can help summarize and organize this information. But evaluating the quality of the AI output
is only one part of the problem.

The stronger question is:

Did the AI-supported process preserve the information necessary for sound professional judgement?

This shifts the focus from simply measuring AI performance to evaluating
AI-supported work.

EvalCommunity adaptation

5. Applying the lesson to an M&E workflow

The following is an EvalCommunity educational adaptation.
It is not a workflow tested in the original healthcare study.

Imagine an evaluation team conducting interviews and focus groups and using AI to support
qualitative synthesis.

Original evidence → AI-assisted synthesis → Human verification → Interpretation → Evaluation finding

At each stage, the team can ask what happened to the original evidence.

6. What should evaluators examine?

These dimensions are an EvalCommunity educational adaptation and are not indicators reported in the original study.

7. The evidence traceability test

One practical test for an AI-assisted M&E workflow is whether an evaluator can move backwards
from a finding to the evidence supporting it.

Evaluation finding ← Human interpretation ← AI synthesis ← Original evidence

If a finding cannot be connected to its underlying evidence, the AI workflow may have created
a traceability problem even if the final text appears clear and convincing.

This connects directly with broader EvalCommunity guidance on

using AI responsibly in evaluation practice
.

8. Human-in-the-loop means more than proofreading

A weak workflow might look like:

AI generates → Human quickly approves → Final report

A stronger workflow is:

AI generates → Human questions → Source verification → Correction → Interpretation → Documented judgement

The evaluator is not simply checking grammar. Human judgement remains important for interpreting
context, recognizing uncertainty, identifying contradictory evidence and determining whether
the evidence supports the conclusion.

This principle is also reflected in EvalCommunity’s

Principles for AI Use in Evaluation
.

9. The “clean summary” problem

Qualitative evidence can contain contradictions, uncertainty, unusual experiences,
minority perspectives and contextual explanations.

AI-generated summaries often aim to be concise and coherent. But in evaluation,
a more coherent summary is not necessarily a more faithful representation of the evidence.

Evaluation reminder: Concise and coherent does not automatically mean high quality.

Sometimes the most important finding is precisely that stakeholders disagreed,
evidence was contradictory, or a small group reported an important issue.

10. How could an M&E team test the workflow?

1. Select cases. Include routine, complex, contradictory and unusual cases.
2. Generate AI summaries. Use the workflow intended for real work.
3. Review the original evidence. Have evaluators examine the source material independently.
4. Compare. Examine the original evidence, AI synthesis and final human interpretation.
5. Record discrepancies. Identify omissions, errors, overgeneralizations, loss of nuance and incorrect interpretations.
6. Improve the workflow. Adjust prompts, review procedures, escalation rules or the appropriate role of AI.

EvalCommunity’s

AI Tool Evaluation Protocol for M&E

provides a related practical approach to testing and documenting AI performance.

11. Possible indicators

As an educational adaptation, EvalCommunity could use indicators such as:

• Evidence traceability rate — percentage of AI-assisted findings that can be traced to supporting evidence.
• Critical omission rate — percentage of reviewed outputs containing a consequential omission.
• Human correction rate — percentage of AI outputs requiring substantive modification.
• Contradiction preservation rate — percentage of identified contradictory findings retained in the synthesis.
• Source verification rate — percentage of high-consequence AI findings verified against original evidence.

These are proposed EvalCommunity indicators, not indicators reported in the original healthcare study.

12. What does the original study actually prove?

The evidence boundary matters.

The study provides evidence about how 16 GPs participated in co-design activities and how
participants envisioned AI-supported documentation, including issues around professional autonomy,
human–AI collaboration and the role of human supervision.

It does not establish that AI documentation improves patient outcomes,
improves evaluation quality, reduces documentation time by a particular percentage,
or reliably preserves every important piece of information.

Those questions require additional empirical evidence.

EvalCommunity framework

13. The AI Evidence Chain

Building on the lesson of this documented study, EvalCommunity proposes a simple way
for M&E professionals to think about AI-supported evidence work:

SOURCE → PROCESS → REVIEW → INTERPRETATION → DECISION → TRACEABILITY

The further an AI-supported finding moves from the original evidence,
the more important traceability and human judgement become.

Take the next step

Learn how to use AI in M&E without losing the human judgement behind good evaluation.

EvalCommunity Academy’s AI in Monitoring & Evaluation Certificate
takes these ideas into practical workflows covering evaluation design, evidence synthesis,
qualitative analysis, reporting, ethics, bias, transparency and human oversight.

Explore the AI in M&E Certificate →

Related EvalCommunity resources

• How to Use AI in Evaluation Practice
  
• AI Tool Evaluation Protocol for M&E
  
• Principles for AI Use in Evaluation
  
• The 7 Layers of AI for M&E Professionals