EvalCommunity Academy · Module III Application Lab

Build, Connect, Debug & Observe Your M&E AI Agent

A practical 35-step laboratory for turning your Module II design into a working, testable and observable AI-agent system for evaluation and M&E.

Build Make.com Debug Observe Human review

Core principle

Build the system — not just the prompt.

A working M&E agent requires appropriate inputs, context, model instructions, tools, workflow logic, validation, observability and human control. A convincing answer alone is not evidence of reliability.

35-step build pathway

1–10
Translate design

11–19
Prototype & test

20–30
Implement

31–32
Observe & debug

33–35
Complete & document

How to use this laboratory

Use the AI agent you designed in Module II.
Use synthetic or approved non-sensitive data while learning.
Complete the steps in order.
Record failures rather than hiding them.
Do not move to deployment simply because the workflow runs.
Preserve this record for Module IV verification and testing.

Retrieve your Module II design

Bring forward the specification you completed in Module II.

REMEMBER — Tool: Use your saved Module II Application Lab, Word, Google Docs, Notion or another approved document. Do not redesign the purpose during implementation.

Restate the M&E problem

Write the actual methodological problem the system is intended to support.

REMEMBER: “Build an AI agent” is not the problem. The evaluation task is the problem.

Confirm the agent goal

State the outcome the system should produce.

Define the methodological boundary

Separate AI tasks from professional judgement.

AI CAN

AI CAN ASSIST WITH

HUMAN MUST DECIDE

AI SHOULD NOT DECIDE

REMEMBER: Human oversight should be designed into the workflow, not added as an afterthought.

List the required inputs

Identify exactly what the system needs before it can work.

REMEMBER — Tool: Use your data dictionary, indicator register, evaluation protocol or workflow specification.

Identify evidence and context

Specify the sources that the agent may rely upon.

REMEMBER: The model should not invent missing evidence. Define what happens when evidence cannot be found.

Define the output

Specify exactly what the agent should return.

Identify tools

Decide which tasks should use deterministic or external tools rather than the model alone.

REMEMBER: Do not use an LLM for a task that is better handled deterministically.

Define human-control points

Identify where the evaluator must inspect, approve or reject the result.

Define the stop conditions

Tell the system when it must stop, ask for clarification or escalate.

REMEMBER: A reliable system knows when it should not continue.

Choose your prototype environment

REMEMBER — Where: Open the chosen AI interface and build the smallest prototype that demonstrates the core task before connecting automation.

Build the first prototype

Turn the design into a reusable instruction.

REMEMBER: The prompt defines the model task. It does not replace workflow logic, retrieval, tools or human review.

Run a simple test

REMEMBER: Establish a baseline before introducing complexity.

Run a complex test

Run an incomplete-data test

REMEMBER: The correct response to missing information may be to stop and request information—not to guess.

Run a contradiction test

REMEMBER: Contradictory evidence should be surfaced, not silently resolved by the model.

Compare output with expected behaviour

Record the first failure

REMEMBER: Do not simply write “AI error”. Identify whether the problem was input, instruction, retrieval, model output, tool or workflow logic.

Refine and retest the prototype

Decide what belongs in automation

Separate the prototype from the operational workflow.

REMEMBER: Automate repeatable process logic; do not automate professional judgement simply because it is technically possible.

Create the Make.com scenario

Open Make.com and create a new scenario/workflow.

REMEMBER — Where: Make.com → Scenario/workflow builder → New scenario. The exact interface may change; focus on the architecture rather than memorising menu labels.

Configure the trigger

REMEMBER — Tool: Select the Make.com trigger that corresponds to your architecture. Test that it actually starts the scenario.

Connect the data source

REMEMBER: Connect only the data required by the agent. Consider privacy and access permissions.

Map the M&E fields

REMEMBER: Check definitions, units, timeframes and sources. Field mapping is an M&E methodological task, not merely a technical task.

Connect the AI/model step

REMEMBER — Where: In your automation platform, add the appropriate AI/model action and explicitly pass the instructions and context required by the model.

Add tools and deterministic operations

REMEMBER: Calculation → Interpretation → Evaluative judgement are different tasks. Do not let the model silently replace deterministic calculation.

Add retrieval or evidence access

REMEMBER: Retrieval is not the same as memory. The automated workflow must explicitly provide the evidence/context required for the task.

Add filters and conditional routing

REMEMBER — Tool: Use Make.com filters/routers or equivalent workflow logic for deterministic conditions.

Add a validation gate

REMEMBER: Example gates: required fields present, calculation completed, evidence source present, output schema valid and no critical failure detected.

Add the human approval route

REMEMBER: A human approval step should have an explicit condition and action—not simply a statement saying “human reviews”.

Run the complete workflow

REMEMBER — Where: Run the Make.com scenario manually before scheduling it. Inspect each module's result.

Inspect observability and execution history

REMEMBER: Observability means being able to inspect what happened during execution—not merely seeing the final answer.

Generate an M&E document if required

REMEMBER: Generated documents remain subject to human verification before dissemination.

Test visual/image analysis where appropriate

REMEMBER: Cropping or tool-assisted visual analysis can improve inspection, but it does not guarantee correct interpretation.

Complete the implementation record

Summarise what you built, what failed, what changed and what remains to be verified.

REMEMBER: This record is evidence of implementation—not proof that the agent is reliable. Module IV will test that question.

Final implementation gate

Is your agent ready for verification?

0 / 24 checks confirmed.

Save your Module III work

The controls below work locally in the learner's browser. No information is transmitted by this laboratory.

Module IV — Verify, evaluate and improve

You have built and documented the system. Module IV now asks the harder question: What evidence do we have that it works reliably?

" ); w.document.close(); window.setTimeout(function(){ w.focus(); w.print(); },500); }); show("Implementation record generated.
Preserve your tests and failure evidence for Module IV.",true); $("m3_result").scrollIntoView({ behavior:"smooth", block:"start" }); } function copyText(text){ if(navigator.clipboard && window.isSecureContext){ navigator.clipboard.writeText(text).then(function(){ show("Copied. The implementation record is on your clipboard.",true); }).catch(function(){ fallbackCopy(text); }); }else{ fallbackCopy(text); } } function fallbackCopy(text){ var area=document.createElement("textarea"); area.value=text; area.style.position="fixed"; area.style.left="-9999px"; area.style.top="0"; document.body.appendChild(area); area.focus(); area.select(); var ok=false; try{ ok=document.execCommand("copy"); }catch(e){ ok=false; } document.body.removeChild(area); if(ok){ show("Copied. The implementation record is on your clipboard.",true); }else{ show("Copy is unavailable in this browser. Select the record manually.",false); } } /* WORKED EXAMPLE */ function example(){ var fields={ m3_design:"Indicator Performance Review Agent: review quarterly indicator data, calculate deviations, retrieve contextual evidence, identify contradictory evidence and prepare a structured review brief for evaluator review.", m3_problem:"The evaluator needs to identify unusual indicator performance and relevant contextual evidence without manually checking every routine record.", m3_goal:"Produce an evidence-linked review brief that identifies deviations, possible explanations, contradictions and issues requiring evaluator investigation.", m3_can:"Retrieve and organise evidence; calculate simple defined metrics using approved deterministic tools; summarise supplied evidence; flag unusual patterns; structure a review brief.", m3_assist:"Compare evidence, identify possible explanations, identify contradictions and draft a structured analytical summary.", m3_human:"Interpret contextual meaning; assess competing explanations; make evaluative judgements; approve conclusions and recommendations.", m3_not:"Make final causal claims, make consequential evaluative judgements or silently resolve contradictory evidence.", m3_inputs:"Indicator ID, definition, target, actual, reporting period, previous period and source.", m3_evidence:"Monitoring report, programme report, approved indicator register and contextual evidence.", m3_output:"Indicator; calculation; evidence; source; interpretation; contradiction; uncertainty; reviewer action.", m3_tools:"Spreadsheet/calculation tool for numerical calculations; document repository for evidence; Make.com for workflow routing.", m3_control:"Evaluator reviews flagged cases and approves substantive interpretation before reporting.", m3_stop:"Missing critical data, failed calculation, contradictory evidence without resolution, unsupported causal claim or consequential judgement.", m3_prompt:"You are an M&E evidence-review assistant. Use only supplied evidence. Separate calculation from interpretation and judgement. Do not invent evidence. Identify contradictions and uncertainty. Return indicator, calculation, evidence, source, interpretation, contradiction, uncertainty and reviewer action.", m3_simple:"Three complete indicators with clear targets and approved sources.", m3_complex:"Twelve indicators across multiple reporting periods with missing contextual information.", m3_incomplete:"Target value is missing. Expected behaviour: do not calculate; flag missing information.", m3_contradiction:"Monitoring report attributes a decline to implementation disruption while another approved source indicates no disruption. Expected behaviour: flag contradiction for evaluator review.", m3_compare:"Simple case passed. Complex case initially missed one outlier. Incomplete case correctly requested information. Contradiction case required an additional contradiction rule.", m3_failure:"The prototype interpreted a deviation as programme failure. Failure point: model interpretation. Root cause: the instruction did not distinguish observed change from causal explanation.", m3_refine:"Added an explicit methodological boundary and alternative-explanation requirement, then reran the original test.", m3_automation:"Trigger → retrieve indicator record → calculate → retrieve evidence → model interpretation → validation → route → human review → output.", m3_make:"Create a scenario representing the architecture above.", m3_trigger:"Scheduled run or approved new indicator record.", m3_source:"Approved indicator spreadsheet/data source.", m3_mapping:"Indicator_ID → context; Target → calculation; Actual → calculation; Period → context; Source → evidence; Review_Status → human gate.", m3_model:"AI/model step receives explicit instructions, calculated result and approved evidence.", m3_toolstep:"Calculate deviation deterministically before asking the model to interpret it.", m3_retrieval:"Retrieve approved monitoring/context documents and preserve source references.", m3_routing:"Complete evidence → continue; missing evidence → request information; contradiction → evaluator review; critical error → stop.", m3_validation:"Required fields present; calculation completed; evidence source present; output structure valid; no critical failure.", m3_approval:"Evaluator reviews evidence, interpretation, contradictions and proposed reviewer action.", m3_run:"Run the scenario manually and inspect every stage before scheduling.", m3_observe:"Record input, model instruction/version, retrieved evidence, model output, validation result, routing decision, errors and human review status.", m3_document:"Word / DOCX", m3_docverify:"Verify values, calculations, source references, tables, formatting and conclusions before sharing.", m3_image:"No image analysis required", m3_imageverify:"No visual component is required for this agent.", m3_summary:"The prototype was translated into a controlled workflow with deterministic calculation, evidence retrieval, validation, conditional routing and evaluator approval. Remaining reliability questions move to Module IV." }; Object.keys(fields).forEach(function(k){ var e=$(k); if(e)e.value=fields[k]; }); root.querySelectorAll(".m3_obs").forEach(function(e){ e.checked=true; }); show("Worked M&E example loaded.
Modify it to create your own implementation.",true); updateCount(); $("m3_design").scrollIntoView({ behavior:"smooth", block:"center" }); } /* SAVE */ function save(){ try{ var data={ fields:{}, observations:[], checks:[] }; root.querySelectorAll("textarea,select").forEach(function(e){ data.fields[e.id]=e.value; }); data.observations=checked("m3_obs"); root.querySelectorAll("#m3_checks input[type='checkbox']").forEach(function(e){ data.checks.push(e.checked); }); localStorage.setItem("ecm3_module3",JSON.stringify(data)); show("Saved locally.
Your work was saved in this browser.",true); }catch(e){ show("Local saving is unavailable in this browser.",false); } } /* RESTORE */ function restore(){ try{ var raw=localStorage.getItem("ecm3_module3"); if(!raw){ show("No saved Module III record was found in this browser.",false); return; } var data=JSON.parse(raw); Object.keys(data.fields || {}).forEach(function(k){ var e=$(k); if(e)e.value=data.fields[k]; }); root.querySelectorAll(".m3_obs").forEach(function(e){ e.checked=(data.observations || []).indexOf(e.value)!==-1; }); root.querySelectorAll("#m3_checks input[type='checkbox']").forEach(function(e,i){ e.checked=!!((data.checks || [])[i]); }); updateCount(); show("Restored.
Your saved Module III work has been loaded.",true); }catch(e){ show("The saved record could not be restored.",false); } } /* RESET */ function reset(){ if(!window.confirm("Reset all Module III Application Lab fields?")){ return; } root.querySelectorAll("textarea").forEach(function(e){ e.value=""; }); root.querySelectorAll("select").forEach(function(e){ e.selectedIndex=0; }); root.querySelectorAll("input[type='checkbox']").forEach(function(e){ e.checked=false; }); $("m3_result").innerHTML=""; $("m3_result").style.display="none"; $("m3_status").innerHTML=""; $("m3_status").style.display="none"; updateCount(); } $("m3_generate").addEventListener("click",generate); $("m3_example").addEventListener("click",example); $("m3_save").addEventListener("click",save); $("m3_restore").addEventListener("click",restore); $("m3_reset").addEventListener("click",reset); updateCount(); show("Lab ready.
Begin with your Module II design specification.",true); })();