AI in Evaluation: Good Practice Guidelines for Practitioners – UK Evaluation Society (UKES)
AI in Evaluation: Good Practice Guidelines for Practitioners
An examination of the UK Evaluation Society’s professional guidance on responsible AI use in evaluation.
About This Case Study
This EvalCommunity Academy case study examines the publication AI in Evaluation: Good Practice Guidelines for Practitioners, co-produced by the UK Evaluation Society (UKES) AI Working Group.
It presents the publication’s purpose, four core principles, implementation framework, risk-based approaches, documentation requirements and recommendations for professional learning.
Source attribution: This is an educational case study developed by EvalCommunity Academy based on the attached UK Evaluation Society document. It is not an official UKES publication.
1. Introduction and Purpose
Artificial intelligence offers significant opportunities to enhance evaluation practice throughout the evaluation lifecycle.
According to the UK Evaluation Society guidelines, AI can support:
- Evaluation design and methodology development.
- Data analysis.
- Evidence synthesis.
- Accessible reporting.
However, these opportunities come with responsibilities. The credibility of evaluation depends on maintaining rigorous standards of evidence, transparency and accountability.
The guidelines establish recommended expectations for responsible AI practice while supporting innovation and learning within the evaluation community.
Evaluation quality and purpose should drive decisions about AI adoption, not technological capability alone.
2. Scope and Application of the Guidelines
The UKES guidelines cover AI applications throughout four phases of the evaluation lifecycle:
The principles are tool-agnostic. They cover publicly available AI platforms, including large language models, as well as bespoke AI solutions developed for specific organisations or evaluation purposes.
The guidelines apply whether AI use is extensive or limited, primary or supplementary.
They are intended for evaluation practitioners working in different organisational contexts, including individual consultants, small NGOs, government departments, healthcare organisations, academic institutions and private sector evaluators.
The document also states that these principles should complement existing evaluation standards and organisational policies on data protection, information security, ethics and professional conduct.
3. The Four Core Principles
The UK Evaluation Society identifies four principles that provide the foundation for responsible AI adoption in evaluation.
Transparency, Accountability and Competence
Evaluators should maintain transparency, develop appropriate competence and take full responsibility for AI-supported evaluation work.
The guidelines identify three main areas:
- AI use should be transparent to stakeholders, with documentation enabling scrutiny and accountability.
- Evaluators should understand AI capabilities and limitations, acknowledge uncertainty and seek technical expertise when necessary.
- Evaluation professionals should contribute to collective learning by documenting experiences and sharing lessons.
- Can I explain to stakeholders how AI was used and trace findings to underlying evidence?
- Do I possess sufficient competence to use this tool responsibly and acknowledge its limitations?
Source: UKES guidelines, page 3.
Human Control and Proportionate AI Use
Evaluators must maintain full control and accountability over AI outputs, with continuous oversight of how information is generated and used throughout the evaluation process.
The guidelines emphasise that:
- Meaningful human oversight must be maintained at all critical decision points.
- AI should augment and assist specific evaluation tasks without replacing professional judgement.
- AI deployment should be guided by evaluation requirements rather than technological capability alone.
- Safeguards should be proportionate to the scale, complexity and risks of AI applications.
- Does this AI application genuinely serve core evaluation purposes?
- Are planned safeguards proportionate to the risks?
- Do I retain meaningful control over key evaluation decisions?
Source: UKES guidelines, page 4.
Active Risk Management and Harm Prevention
Evaluators must actively identify, assess and manage risks throughout AI deployment, implementing proportionate safeguards to prevent or minimise harm.
The guidelines identify several areas requiring attention:
- Bias and discrimination.
- Privacy breaches.
- Environmental impact.
- Threats to evaluation credibility.
- Misrepresentation of vulnerable populations, minority groups and people with limited voice in evaluation processes.
Data protection and confidentiality requirements must be met, with explicit consent obtained where AI processing involves personal or sensitive information.
- What potential risks could this AI application introduce?
- How will I systematically assess and monitor these risks?
- What safeguards and mitigation measures should I implement?
- How will I engage stakeholders about AI use?
Source: UKES guidelines, page 5.
Quality Assurance and Verification
AI-enhanced outputs should meet the same standards of accuracy and reliability as traditional evaluation methods.
The guidelines recommend:
- Systematic verification and validation of AI-generated content before use.
- Verification approaches proportionate to AI’s role in evaluation conclusions.
- Sampling-based verification where comprehensive manual verification is impractical.
- Cross-validation with alternative methods.
- Systematic quality control processes.
- How will I verify that AI outputs are accurate, complete and reliable?
- Are my verification processes practical and sufficient to maintain professional evaluation standards?
Source: UKES guidelines, page 6.
4. The UKES Implementation Framework
Before implementing AI in an evaluation process, the guidelines recommend a systematic approach to determining whether AI use is justified and appropriate.
The central assessment question:
Can demonstrated application of principles for AI use in evaluation sufficiently mitigate identified risks to render the AI deployment acceptable for the intended evaluation purpose?
The framework operationalises the Active Risk Management principle through three elements:
Risk Identification
Consider material risks from AI use in the specific evaluation context, including risks to evaluation quality, stakeholder trust, participant privacy, bias introduction and evaluation credibility.
Principle Application
Address each identified risk through concrete application of relevant ethical principles and document how these principles translate into practical safeguards and controls.
Residual Risk Assessment
Acknowledge remaining risks after applying the principles and assess whether they are acceptable given the evaluation’s purpose, importance, sensitivity and potential consequences.
Source: UKES guidelines, page 7.
5. Proportionate Risk-Based Implementation
The UKES document distinguishes between three levels of AI application risk.
| Risk level | Examples in the guidelines | Recommended approach |
|---|---|---|
| Lower risk | Administrative support, basic document formatting and simple summarisation. | Basic verification, simple documentation and standard disclosure. |
| Medium risk | Supporting data analysis, stakeholder communication and evidence synthesis. | Systematic verification, clear documentation, appropriate stakeholder disclosure and planned quality assurance. |
| Higher risk | Direct influence on evaluation conclusions, processing sensitive data and applications affecting vulnerable populations. | Enhanced validation, multiple methods, expert review, comprehensive documentation and intensive human oversight. |
Source: UKES guidelines, page 8.
The guidelines also recommend revisiting risk assessments when circumstances change. Where risks are unacceptable, evaluators should consider modifying the AI application, introducing additional safeguards or avoiding AI use for that specific purpose.
6. Professional Development Considerations
The UKES guidelines identify five areas for building competence in responsible AI adoption:
- Understanding AI capabilities and limitations relevant to evaluation applications.
- Developing skills in AI tool assessment, including bias detection and reliability evaluation.
- Learning appropriate verification and validation approaches for different AI applications.
- Building knowledge of data protection and ethical requirements for AI use.
- Engaging with specialist expertise through collaboration and professional networks.
7. Contractual Considerations
The guidelines recommend that organisations may wish to address AI use in contractual arrangements to establish clear expectations, accountability and legal compliance.
Key considerations include:
- Specifying permitted AI applications and restrictions.
- Establishing ethical obligations and verification responsibilities.
- Defining data protection requirements.
- Ensuring consistent standards across team members and stakeholders.
- Maintaining alignment with organisational policies and regulatory requirements.
Source: UKES guidelines, page 9.
8. Documentation and Disclosure Examples
The publication provides sample approaches for disclosing AI use in evaluation reports and proposals.
Standard Disclosure for Evaluation Reports
“This evaluation used artificial intelligence tools to support [specific functions, e.g., data analysis, document synthesis, stakeholder communication]. All AI-generated outputs were verified through [specific verification methods, e.g., human review, cross-validation with alternative sources, expert consultation]. The evaluation team retained full responsibility for methodology, analysis, findings, and recommendations.”
The UKES example also recommends specifying tools used, identified limitations, verification processes and relevant organisational policies.
Disclosure in Evaluation Proposals
“This evaluation proposes to use AI tools for [planned applications, e.g., qualitative data analysis, evidence synthesis, report drafting] with associated quality assurance measures including [planned verification approaches].”
The example further addresses documenting changes to AI applications and maintaining human oversight of key evaluation decisions.
Risk-Proportionate Disclosure
The guidelines provide different disclosure approaches for lower-, medium- and higher-risk applications:
- Lower risk: Specify administrative AI tasks, confirm output verification and retain responsibility for evaluation quality.
- Medium risk: Specify AI applications, systematic verification methods, human oversight and quality assurance measures.
- Higher risk: Specify comprehensive safeguards, multiple validation methods, enhanced human oversight and additional protections.
Recommended Documentation Practices
The UKES publication identifies three documentation contexts:
- Professional peer review: Document tools, verification approaches, quality assurance measures, lessons learned and limitations.
- Client relationships: Discuss planned AI applications, safeguards, risk assessments and implications for evaluation processes and timelines.
- Internal quality assurance: Maintain comprehensive records of AI applications, verification, quality control and continuous monitoring.
Source: UKES guidelines, pages 9–10.
9. Supporting Professional Learning
The guidelines recognise that AI technology and evaluation practice continue to evolve.
The UK Evaluation Society identifies four ways the evaluation community can support professional learning:
- Share experiences and lessons learned from AI implementation.
- Develop case studies demonstrating ethical and effective AI applications.
- Contribute to ongoing refinement of professional guidance and standards.
- Participate in cross-sector learning about responsible AI use in research and evaluation.
10. Reflection and Knowledge Check
The following questions are based on the principles and recommendations presented in the UKES document.
View answer
Evaluation quality and purpose, rather than technological capability alone.
View answer
- Transparency, accountability and competence.
- Human control and proportionate AI use.
- Active risk management and harm prevention.
- Quality assurance and verification.
View answer
- Risk identification.
- Principle application.
- Residual risk assessment.
View answer
Consider modifying the AI application, implementing additional safeguards or avoiding AI use for that specific purpose.
View answer
AI-generated content should be systematically verified and validated before use in evaluation work. Verification approaches should reflect the application’s role in evaluation conclusions.
11. Key Messages for Evaluation Professionals
The UK Evaluation Society guidelines provide a professional foundation for responsible AI adoption throughout the evaluation lifecycle.
The publication brings together transparency, human control, active risk management and quality assurance as essential principles for maintaining evaluation credibility while enabling innovation.
The central professional responsibility
Evaluators must maintain transparency, human control, accountability and verification when using AI in evaluation.
12. Original Publication and Source
Title: AI in Evaluation: Good Practice Guidelines for Practitioners
Publisher: UK Evaluation Society
Produced by: UK Evaluation Society AI Working Group
Publication date: November 2025
Version: 1.0
Official website: evaluation.org.uk
All substantive information in this case study is based on the supplied UKES publication. The reflection questions are derived from the document’s stated principles and key questions.
Explore AI in Monitoring and Evaluation
Explore the EvalCommunity Academy courses and professional learning resources for Monitoring and Evaluation and International Development practitioners.
