What Is an AI Indicator Tracking and Early-Warning Agent?

An AI indicator tracking and early-warning agent is a controlled M&E workflow that reads an approved results framework, targets, reporting dates, actual values, and evidence references. A deterministic checker calculates and validates the records; an AI assistant then explains the findings and drafts follow-up questions. Humans remain responsible for verification, corrections, communication, and management decisions.

Before You Start

This tutorial is designed for complete beginners. You do not need to be a programmer, but you will create ordinary folders and files and run one Python command.

What you need

• A Windows, macOS, or Linux computer.
• Permission to install or use Python 3.
• The downloadable starter pack supplied with the lesson.
• An approved AI assistant with access to the project folder, or a manual copy-and-paste workflow.
• Fictional or approved non-sensitive data for practice.

What the agent can and cannot do

Step 1 — Choose a Suitable Indicator-Tracking Task

Do not begin with “monitor the whole programme.” Choose one repeated, rule-based review that produces a draft for a human. Use the five tests below.

1. Does the task repeat?

Good example: Every quarter, the M&E team receives the same indicator tracking table from programme teams. Poor example: A one-time strategic review of whether the programme model should change.

2. Are the rules already defined?

The organisation should know which targets are approved, whether higher or lower values are better, when reports are due, and what evidence is required. If staff disagree, resolve the methodology before automation.

3. Can the result be checked?

“Tell us whether the programme is doing well” is not checkable. “Compare actual results with approved targets and flag records below 70%” is checkable because the target, formula, and rule can be reviewed.

4. Can permissions remain limited?

The first version should read approved files and write draft outputs only. It should not edit the tracking table, update a donor portal, send email, or change official targets.

5. Is there a clear human review point?

The workflow should stop when evidence conflicts, a target source is disputed, an indicator direction is unclear, or management action is required.

Step 2 — Create the Project Folder

A project folder is an ordinary folder on your computer. It keeps the instructions, source data, checker, and outputs together.

Easiest option: use the starter pack

1. Download the ZIP file supplied with the tutorial.
2. Open Downloads.
3. On Windows, right-click and choose Extract All. On macOS, double-click the ZIP.
4. Move the extracted folder into Documents.
5. Open it and confirm that you see README.md, TASK.md, STATUS_RULES.json, source-data, tools, and outputs.

Create it manually

ai-indicator-tracking-early-warning-agent-starter-pack/
├── README.md
├── TASK.md
├── INDICATOR_RULES.md
├── STATUS_RULES.json
├── LOOP_INSTRUCTIONS.md
├── PROGRESS.md
├── source-data/
│   ├── results-framework.md
│   ├── indicator-reference-sheet.csv
│   ├── approved-targets.csv
│   └── indicator-tracking.csv
├── tools/
│   └── check_indicators.py
└── outputs/
    ├── check-summary.json
    ├── indicator-check-results.csv
    ├── indicator-status-report.md
    ├── indicators-needing-attention.csv
    └── partner-questions.md

Step 3 — Prepare the Results Framework

A results framework connects expected results with indicators, baselines, targets, data sources, means of verification, and other metadata. Open source-data/results-framework.md and confirm that every indicator belongs to a stated result.

Before continuing, confirm

• Every indicator has a unique ID.
• Every indicator is linked to an output, outcome, or other defined result.
• The indicator measures something the programme can reasonably monitor.
• Assumptions and risks are considered separately from indicator performance.

Step 4 — Check the Indicator Definitions

Open source-data/indicator-reference-sheet.csv in a spreadsheet application. Each row describes one indicator.

Definition checks before automation

• The numerator, denominator, population, and exclusions are clear for percentage indicators.
• The same person, household, or activity is not double-counted unless repeated counts are permitted.
• The reporting frequency matches the target schedule.
• The direction of improvement is explicit.
• The means of verification exists and is accessible to authorised reviewers.
• The unit has a sensible range: percentages normally remain between 0 and 100; counts and days normally are not negative.

Step 5 — Connect Baselines, Approved Targets, and Actual Results

The starter pack separates approved-targets.csv from indicator-tracking.csv. The approved-target file is the source of truth. The tracking table contains the target copied into the current report, allowing the checker to detect a mismatch.

Achievement and variance

Higher is better:
achievement % = actual result ÷ approved target × 100
Example: 82 ÷ 80 × 100 = 102.5%
Lower is better:
achievement % = approved target ÷ actual result × 100
Example: 5 days ÷ 7 days × 100 = 71.4%
Variance value = actual result − approved target

For a higher-is-better indicator, a positive variance is usually favourable. For a lower-is-better indicator, a negative variance is usually favourable. Always interpret variance with the direction.

Step 6 — Define Reporting Periods and Due Dates

A result is meaningful only when attached to the correct period. The checker compares the label, start date, end date, due date, reported date, and the rules file’s as-of date.

• Period label: the cycle, such as 2026-Q2.
• Period start and end: the dates covered by the result.
• Report due date: the agreed submission deadline.
• Reported date: when the result was submitted.
• As-of date: the date used to determine whether a blank result is overdue.

Step 7 — Approve the Traffic-Light and Alert Rules

Open STATUS_RULES.json. The thresholds are explicitly marked as illustrative. An operational workflow must use rules approved by the organisation, programme, or donor requirements.

Do not confuse status with alert severity

The traffic light describes performance assessability or progress. The alert label describes the type of follow-up:

• Critical: the record cannot be interpreted because of a fundamental structural problem.
• Error: a clear data, date, period, target, duplicate, or range problem.
• Warning: a timeliness or evidence concern that needs checking.
• Review: a performance or judgement issue requiring an authorised human decision.

Step 8 — Run the Deterministic Checker

The Python checker performs the repeatable validation and calculations. The AI should not be the only calculation mechanism because language-model outputs can vary.

Install or confirm Python

Windows:
py --version
macOS or Linux:
python3 --version

If the command fails, install Python 3 from the official Python website or use an organisation-managed installation.

Open the terminal in the project folder

Run the checker

Windows:
py tools/check_indicators.py
macOS or Linux:
python3 tools/check_indicators.py

Expected result

Rows reviewed: 12
Records needing attention: 7
Unique indicators needing attention: 6
Green: 5
Amber: 3
Red: 2
Grey: 2

The checker writes exact counts, issue counts, a run timestamp, individual input fingerprints, and a combined input fingerprint to outputs/check-summary.json.

Step 9 — Understand the Outputs

• check-summary.json: exact counts, rules version, timestamps, and input fingerprints.
• indicator-check-results.csv: every reviewed indicator-period record with calculations, status, severity, and alert codes.
• indicators-needing-attention.csv: only records that are not Green or contain an alert.
• indicator-status-report.md: a deterministic baseline report that the AI may expand without changing facts.
• partner-questions.md: draft neutral follow-up questions for human review.

Key alert codes

Step 10 — Ask the AI to Explain the Results

Open the project folder in an approved AI environment that can read the files. In Claude Code, use a permission mode that asks before edits and commands, and review the permissions before approving changes.

Run the indicator tracking and early-warning review for this project.
Before writing:
1. Read TASK.md, LOOP_INSTRUCTIONS.md, PROGRESS.md and INDICATOR_RULES.md.
2. Read STATUS_RULES.json.
3. Read the indicator reference sheet and approved targets.
4. Read outputs/check-summary.json and outputs/indicator-check-results.csv.
Use exact counts from check-summary.json.
Do not invent or change thresholds, targets, results, dates, evidence or explanations.
Do not modify source-data/ or STATUS_RULES.json.
Update only:
- outputs/indicator-status-report.md
- outputs/partner-questions.md
- PROGRESS.md
Separate machine findings, interpretation, missing evidence and human decisions.
Stop if an approved target, definition, direction or evidence source is unclear.

The AI should add

• A concise executive summary using exact counts.
• A clear explanation of Red, Amber, and Grey records.
• A distinction between performance, reporting, evidence, and structural issues.
• Neutral questions and explicit limitations.

The AI must not add

• Invented causes, dates, results, thresholds, or targets.
• A partner-performance judgement presented as fact.
• An instruction to change official data without approval.
• Counts that differ from the machine summary.

Step 11 — Verify, Escalate, and Update State

A fluent report may still be wrong. Use this acceptance checklist before sharing it.

Update PROGRESS.md

## Last run
- Date and time: [copy from check-summary.json]
- Combined input fingerprint: [copy from check-summary.json]
- Rows reviewed: 12
- Status counts: Green 5; Amber 3; Red 2; Grey 2
- Records needing attention: 7
- Unique indicators needing attention: 6
## Open issues
- Confirm the correct period for IND-01.
- Confirm the approved Q2 target for IND-04.
- Request the overdue IND-03 result.
- Define the means of verification for IND-05.
## Needs human review
- Programme response to the IND-02 Red status.
- Approval of any source-data correction.

Stop and escalate when

• Authorised sources contain conflicting targets.
• The indicator direction or definition is unclear.
• Evidence includes personal, safeguarding, or confidential information.
• A correction would alter an official record.
• A contractual, management, or partner rating decision is being considered.

Step 12 — Test the Workflow Across Two Reporting Periods

The starter pack includes stable Q1 records and deliberately flawed Q2 records. The checker should find all required exercise problems.

Re-run after an authorised correction

1. Make a copy of the original tracking file.
2. Apply only the confirmed correction.
3. Run the checker again.
4. Compare the old and new summary fingerprints and counts.
5. Record who approved the correction and why.

Academy Exercise

1. Run the checker and confirm the expected counts.
2. Locate the two missing results and explain why one is Red and one is Grey.
3. Identify the incorrect period and target mismatch.
4. Explain why IND-05 is Amber although it exceeds target.
5. Identify the substantially below-target indicator without inventing a cause.
6. Use the AI prompt to improve the report and questions.
7. Complete the verification checklist.
8. Update PROGRESS.md with fingerprints, unresolved issues, and decisions required.

Success criteria

• All five deliberate problem types are detected.
• Exact counts match the JSON summary.
• No source data are changed during the first review.
• The report separates performance, reporting, evidence, and structural issues.
• Questions are neutral and actionable.
• At least one issue is explicitly escalated for human judgement.

Common Errors and Troubleshooting

Python is not recognised

Try py --version on Windows or python3 --version on macOS. Restart the terminal after installation.

The checker says a file is missing

Confirm that the ZIP was fully extracted and that the folder names were not changed.

The counts do not match

Restore the original fictional files. A change to a date, target, evidence field, or as-of date can change the status.

A high result is Amber

Check the evidence fields. In the training rules, incomplete evidence caps an otherwise Green result at Amber.

The AI changes the counts

Reject the edit and require check-summary.json to be the only source for totals.

The AI invents a cause

Replace the statement with a neutral question or label it as an unverified hypothesis.

No-Code Alternative

You can reproduce a smaller version in Excel or Google Sheets using locked cells for approved thresholds and formulas for achievement, variance, evidence completeness, late reporting, and status. Protect formula cells, document the version, and manually verify a sample because copied formulas, hidden rows, and manual edits can introduce errors.

Frequently Asked Questions

Does the agent require Claude?

No. The checker works independently. Claude is one possible explanation layer; another approved AI assistant can be used.

Why not ask the AI to calculate everything?

A deterministic checker applies the same explicit rules every run and creates machine-readable evidence. The AI is better used to explain and organise the results.

Are the Green, Amber, and Red thresholds universal?

No. The tutorial thresholds are fictional. Your organisation must approve its own rules.

Can Grey mean poor performance?

No. Grey means the record cannot currently be assessed from valid information.

Should a late report automatically become Red?

Not necessarily. This workflow flags late submission separately. A result still missing after the deadline is Red under the training rules.

Can this workflow prove impact?

No. It tracks results against targets. Impact and causality require appropriate evaluation designs and evidence.

Can I use real beneficiary data?

Only with organisational approval, data minimisation, access controls, and compliance with applicable policy and law. The exercise uses fictional aggregate data.

When should I schedule the workflow?

Only after several stable manual runs, verified outputs, approved permissions, and a clear escalation process.

Official and Further Reading

• OECD: Effective Results Frameworks for Sustainable Development
• Anthropic: Claude Code overview
• Anthropic: Configure Claude Code permissions
• Python: Official downloads

This tutorial uses fictional programme information and illustrative rules. Review the methodology, data-protection requirements, and software instructions before adapting it to an operational environment.