The purpose of this laboratory is not to make the most sophisticated agent possible. It is to design an appropriate system whose purpose, workflow, evidence, inputs, outputs, validation and human responsibilities are explicit.
Design pathway
1 Purpose
→
2 Workflow
→
3 Inputs
→
4 Evidence
→
5 Tools
→
6 Validate
STEP 1 · PURPOSE
Define the agent's purpose
State the M&E task the system will support and define what it will not decide.
REMEMBER — How to do this Define the M&E task, not the technology.
Good: “Identify potentially material indicator changes and prepare evidence for evaluator review.” Weak: “Build an autonomous AI evaluator.”
Tool: Draft this in Word, Google Docs, ChatGPT, Claude or your Module 2 design template.
STEP 2 · WORKFLOW & RISK
Map the workflow and identify risk
Map the real M&E process first. Then identify where errors could have methodological consequences.
REMEMBER — How to map it Map what happens before, during and after each step.
Tools: paper, Word, PowerPoint, Miro, Lucidchart, spreadsheet, ChatGPT or Claude.
For each step ask: What information is needed? What can go wrong? Who checks it?
High risk does not automatically mean “no AI.” It means the design needs stronger controls and clearer human responsibility.
STEP 3 · ARCHITECTURE
Choose the architecture
REMEMBER — Architecture is a design decision Linear: predictable sequence. Retrieval: approved evidence must be found and supplied. Tool-using: external tools are genuinely required. Conditional: different conditions require different paths. Iterative: the system must inspect and revise work. Human-in-the-loop: evaluator approval is required at defined points. Multi-agent: separate specialised components genuinely add value.
Rule: do not use a more complex architecture simply because it sounds more advanced.
STEP 4 · INPUTS
Specify the inputs
Required What must always be present?
Optional What may be missing?
Quality rules What should trigger a stop or review?
REMEMBER — How to specify inputs Do not write only “upload the data.” Define the fields, source, units, definitions, timeframe and quality requirements.
For qualitative work, consider transcript ID, metadata, analytical unit, segmentation and anonymisation. For quantitative work, consider variable definitions, missingness, outliers, denominators and disaggregation.
REMEMBER — Design outputs backwards from verification Ask: “What would an evaluator need to see to verify this output?”
Do not allow the system to hide uncertainty behind a single overall score or conclusion.
STEP 6 · CONTEXT & EVIDENCE
Specify the context and evidence
REMEMBER — Retrieval is not verification A document being retrieved does not prove that an extracted claim is correct.
Design the system so important claims can be traced to their source and location, and contradictory or missing evidence can be flagged.
STEP 7 · TOOLS
Specify the tools
REMEMBER — Use the right tool for the job Use deterministic tools for calculations when reproducibility matters. Use retrieval systems for approved evidence. Use Make.com later for connecting systems and automating the workflow. Use an LLM where interpretation, classification, synthesis or language generation is genuinely useful.
Do not make the model perform a task better handled by a deterministic tool.
STEP 8 · QUALITY GATES
Build validation into the design
REMEMBER — How to build a quality gate A quality gate should create an observable outcome:
PASS → continue FAIL → revise / stop / escalate
Later, these conditions can be implemented through explicit validation logic, filters, routers or validation modules in an automation platform.
STEP 9 · HUMAN CONTROL
Define the methodological boundary
AI CAN Retrieve, organise, classify, compare, calculate with suitable tools and draft defined outputs.
AI CAN ASSIST WITH Coding, synthesis, interpretation, prioritisation and hypothesis generation when reviewed.
HUMAN MUST DECIDE Evaluative judgement, contextual interpretation, causal conclusions and consequential recommendations.
AI SHOULD NOT DECIDE High-stakes methodological, ethical or accountability decisions that cannot be adequately verified.
REMEMBER — Specify the control point Avoid simply writing “human review required.”
Specify what the human checks, when the human checks it and what happens when the human disagrees.
STEP 10 · FAILURE & ESCALATION
Design what happens when the system cannot safely continue
REMEMBER — Good systems know when to stop A reliable agent does not need to produce an answer under every circumstance.
Use: UNCERTAINTY / FAILURE → FLAG → ESCALATE → HUMAN RESOLUTION
STEP 11 · EVALUABILITY
Design the agent so it can later be evaluated
REMEMBER — Design for evaluability now Ask for every important output:
What counts as correct? What evidence demonstrates correctness? What counts as failure? Can the failure be reproduced and inspected?
WORKED EXAMPLE
Indicator Performance Review Agent
Use this example to understand the level of detail expected. Click Load worked example below to place the example into the editable fields.
Purpose: Review quarterly indicator data, identify material deviations, retrieve contextual evidence and prepare a review brief.
Human boundary: The evaluator decides whether the evidence supports an evaluative finding or causal explanation.
Failure rule: Missing or contradictory evidence is flagged rather than silently converted into a conclusion.
Generate your design specification
The generator checks the essential design fields and creates a structured specification for review, copying or printing.
FINAL DESIGN REVIEW
Before you build
0 / 14 design checks confirmed.
A complete specification is not evidence that the eventual agent will work. It is the blueprint that makes implementation, testing and later evaluation possible.