Outcome Harvesting Dashboard with AI
EvalCommunity Academy Practical Tutorial
How to Build an Outcome Harvesting Dashboard with AI
From change narratives to visual insights for monitoring, evaluation and learning
Quick Answer
An outcome harvesting dashboard converts reviewed and verified change narratives into a structured dataset that users can explore by outcome level, country, institution, activity, stakeholder group, evidence source and validation status. AI can help prepare fields, identify patterns, compare evidence and draft narrative summaries. It should not decide whether a change is significant, representative or caused by the programme. Those judgements remain with evaluators and stakeholders.
Where This Tutorial Begins
This tutorial begins after the outcome harvesting process has produced a reviewed dataset. It assumes that outcome statements have already been extracted, formulated, deduplicated and linked to evidence. Where possible, they should also have been verified with relevant stakeholders.
If you are still working with raw reports, interview transcripts, meeting minutes or stakeholder narratives, complete the outcome harvesting tutorial first. It explains how to identify outcomes, test AI extraction, consolidate duplicates, examine contribution evidence and verify findings.
Tutorial Overview
Intermediate
3–5 hours
Spreadsheet plus dashboard platform
Reviewed outcome statements
What You Will Build
- A dashboard-ready outcome harvesting dataset
- A data dictionary with consistent categories
- An overview of outcome levels and change types
- Country and institution comparison views
- An activity-to-change analysis
- A stakeholder voice and representation analysis
- A searchable narrative and evidence explorer
- Evidence-grounded AI summaries with limitations
- A privacy, quality and human-review process
The Central Principle
Do not turn stories into counts and then discard the stories. Every visual summary should remain traceable to the original outcome statement, supporting narrative, evidence source and validation record. The dashboard helps users navigate qualitative evidence; it does not replace evaluative interpretation.
17-Step Workflow
Confirm That the Outcomes Are Ready
A dashboard should not be the first place where extracted outcomes are reviewed. Before importing any records, confirm that each outcome describes a specific change and is connected to evidence.
Minimum readiness conditions:
- Each row represents one distinct outcome.
- Duplicate descriptions of the same change have been consolidated.
- The original source can be located.
- Outcome type and level have been reviewed by a person.
- Contribution claims are separated from evidence that the change occurred.
- Validation status is visible.
- Sensitive details have been removed, protected or access-restricted.
Define the Decisions and Questions
Start with the decisions the dashboard should support. Avoid beginning with the available columns or a list of charts. A useful question describes what the user needs to understand and why it matters.
| Question | Decision or learning use |
|---|---|
| At which outcome levels are changes being recorded? | Assess whether change remains individual or reaches organisations and systems. |
| What patterns differ across countries and institutions? | Identify contexts that require deeper investigation or adaptation. |
| What activities appear alongside different changes? | Explore contribution pathways without claiming attribution. |
| Whose perspectives are represented or absent? | Improve inclusion and target additional data collection. |
| Which outcomes have weak, contradictory or incomplete evidence? | Prioritise verification and evaluative follow-up. |
Create One Row per Distinct Outcome
The dashboard dataset should use a consistent unit of analysis. In most cases, one row should represent one distinct, reviewed outcome. Multiple sources can support that outcome, but they should not automatically become duplicate outcome rows.
| Field | Purpose | Example |
|---|---|---|
| Outcome ID | Unique reference | OH-024 |
| Outcome statement | Concise formulation of what changed | Regional academy revised its training policy to include gender-responsive leadership. |
| Full narrative | Context, sequence and meaning | Longer reviewed account linked to the source. |
| Country | Geographic comparison | Country B |
| Institution | Organisational comparison | Training academy |
| Activity type | Contribution analysis | Technical assistance |
| Stakeholder group | Who experienced the change | Mid-level managers |
| Perspective source | Whose account provides the evidence | Female training participant |
| Outcome level | Level of change | Institutional |
| Change type | Nature of change | Policy or practice |
| Evidence source | Traceability and triangulation | Interview and revised policy |
| Validation status | Verification transparency | Verified |
| Evidence confidence | Strength of supporting evidence | Medium |
| Source reference | Link back to evidence | INT-07, page 4 |
A Note on Multiple Activities and Sources
Do not force a complex outcome into a single activity or source when several contributed. Use a separate contribution table or carefully controlled multiple-value fields. The dashboard should distinguish between the outcome itself, the evidence that it occurred and the programme activities believed to have contributed.
Build the Data Dictionary
A dashboard becomes unreliable when categories are applied differently across countries, analysts or reporting periods. Create a data dictionary before building visuals.
For every field, document:
- Definition
- Permitted values
- Inclusion and exclusion criteria
- Example
- Treatment of missing or uncertain information
- Person responsible for approving revisions
Add Fields for Voice and Representation
The people most frequently mentioned in programme reports are not necessarily the people most affected by the intervention. A representation analysis requires fields that distinguish who experienced a change from who described it.
- Stakeholder expected: Was this group expected to be included according to the stakeholder map or programme design?
- Stakeholder consulted: Was anyone from the group directly involved in data collection?
- Perspective source: Who provided the account supporting the outcome?
- Direct or indirect voice: Is the group speaking directly or being described by another actor?
- Representation status: Present, limited, missing or unknown.
Run a Dashboard-Readiness Review
Before creating charts, inspect the dataset for structural and evidentiary weaknesses.
| Check | Pass condition |
|---|---|
| Source traceability | Every outcome can be linked to supporting evidence. |
| Human review | Every AI-assisted code has been reviewed. |
| Category consistency | Values follow the data dictionary. |
| Validation transparency | Every record has a validation status. |
| Contribution discipline | Association is not presented as attribution. |
| Privacy protection | Sensitive information is removed, masked or access-controlled. |
| Narrative access | Users can move from summaries to the underlying evidence. |
Translate Questions into Dashboard Views
Choose visuals according to the question and the type of evidence. Avoid decorative charts that make a small qualitative dataset appear statistically representative.
| Evaluation question | Suggested view | Required caution |
|---|---|---|
| At which levels are outcomes recorded? | Ordered bar chart | Counts do not measure significance. |
| What patterns differ across countries? | Country-by-change heatmap | Account for unequal documentation volume. |
| Which activities appear with which changes? | Activity-to-change matrix | Do not infer causality. |
| Whose voices are present? | Expected-versus-observed representation table | Protect small or sensitive groups. |
| How strong is the evidence? | Validation and evidence-source view | Confidence labels require clear rules. |
| What do the stories actually say? | Searchable narrative cards or table | Restrict sensitive quotations. |
Build Page 1: Outcome Overview
The first page should orient the user without presenting a single score of programme success.
Recommended components:
- Number of distinct outcomes in the filtered dataset
- Verified, partially verified and unverified outcomes
- Intended and unexpected outcomes
- Outcomes by level
- Outcomes by change type
- Countries, institutions and stakeholder groups represented
- Filters for period, country, institution, stakeholder and evidence status
Build Page 2: Countries and Institutions
This page helps users compare documented patterns while keeping context visible.
- Country-by-change-type heatmap
- Outcome level by country
- Institution-by-outcome matrix
- Intended and unexpected outcomes by context
- Evidence-source and validation distribution
- Narratives linked to every selected pattern
Add a visible note showing the number of documents, interviews or evidence records included for each country or institution. This helps users interpret apparent differences more responsibly.
Build Page 3: Activity-to-Change Analysis
An activity-to-change matrix can reveal recurring contribution patterns. It should be treated as a starting point for interpretation, not proof that one activity caused an outcome.
| Activity type | Behavioural | Relational | Institutional | Policy |
|---|---|---|---|---|
| Training | 18 | 7 | 5 | 1 |
| Dialogue | 8 | 16 | 9 | 4 |
| Technical assistance | 4 | 5 | 14 | 7 |
| Advocacy | 3 | 8 | 10 | 12 |
Build Page 4: Whose Voice Is Represented?
This page compares the groups expected in the programme’s evidence strategy with those actually represented in the outcome narratives.
| Stakeholder group | Expected | Consulted | Present in narratives | Directly represented | Gap |
|---|---|---|---|---|---|
| Women participants | Yes | Yes | 54 | 32 | Moderate |
| Local officials | Yes | Yes | 24 | 18 | Low |
| People with disabilities | Yes | Limited | 4 | 2 | High |
| Informal leaders | Yes | Limited | 3 | 1 | High |
A representation gap is not proof that a group experienced no change. It is evidence that the available dataset may not reflect that group adequately. The correct response may be additional consultation, not an AI-generated assumption.
Build Page 5: Narrative and Evidence Explorer
The explorer preserves the qualitative value of the dataset. It should let authorised users search, filter and inspect the evidence behind every pattern.
Recommended fields:
- Outcome statement
- Full reviewed narrative
- Country and institution
- Stakeholder and perspective source
- Outcome level and change type
- Contribution evidence and alternative factors
- Validation and confidence status
- Source reference and human-review notes
Use AI for Evidence-Grounded Summaries
AI can draft a structured summary of filtered records, but the prompt must constrain it to the evidence provided. The output should distinguish description, interpretation, limitation and follow-up questions.
Act as an M&E qualitative analysis assistant.
Review only the structured outcome records provided. Identify patterns across outcome levels, countries, institutions, activities, stakeholder groups, evidence sources and validation status.
For each finding:
1. State the observed pattern.
2. Report the number of relevant outcome records.
3. List the supporting Outcome IDs.
4. Distinguish evidence from interpretation.
5. Note possible documentation, selection or representation bias.
6. Avoid claims of causality or programme attribution.
7. State when evidence is insufficient or contradictory.
8. Suggest a question for human review.
Do not introduce facts that are not present in the dataset.
Identify Contradictions and Evidence Gaps
A strong qualitative dashboard should not hide disagreement. It should help evaluators locate contradictory accounts, weakly supported outcomes and missing contextual information.
Compare the supplied outcome records across countries, institutions, stakeholder groups and evidence sources.
Identify:
– Findings supported by more than one independent source
– Contradictory accounts of the same change
– Outcomes supported by only one source
– Missing dates, actors, locations or contribution evidence
– Stakeholder groups with limited or indirect representation
– Outcomes that remain unverified
Return a table with:
Issue | Outcome IDs | Evidence sources | Why it matters | Recommended human follow-up
Do not resolve contradictions or fill missing information.
Choose and Configure the Dashboard Tool
The analytical design should be completed before the platform is selected. Choose the tool that fits your organisation’s skills, governance and sharing requirements.
| Tool | Useful starting context | Consider carefully |
|---|---|---|
| Excel | Small dataset, internal prototype, familiar team | Version control, permissions and narrative navigation |
| Power BI | Microsoft environment, structured models, controlled sharing | Licensing, workspace governance and access rules |
| Tableau | Advanced exploration and visual storytelling | Capacity, cost and publication settings |
| Looker Studio | Google-based workflow and accessible prototypes | Data governance and controls for sensitive narratives |
Build order: import data, standardise fields, create supporting tables, add filters, construct overview views, connect narratives, test every figure against the source records, and review the prototype with intended users.
Protect Sensitive Qualitative Data
Outcome narratives may contain personal, political, operational or institutional information that creates risk if exposed. This is especially important in gender equality, governance, humanitarian, conflict and security-sector programmes.
- Remove or pseudonymise names and direct identifiers.
- Review indirect identifiers that could reveal a person through context.
- Suppress or combine small categories when disclosure risk is high.
- Separate public, donor and internal dashboard versions.
- Restrict narrative-level access by role.
- Use only AI systems approved for the data classification involved.
- Do not upload raw sensitive transcripts to public AI tools without authorisation and appropriate safeguards.
- Review generated summaries for disclosure, stereotyping and unsupported inference.
Validate, Document and Publish
Review the dashboard with the M&E team, programme staff and selected stakeholders before it is used for reporting or decision-making.
Validate:
- Category definitions
- Unexpected or sensitive patterns
- Country and institution comparisons
- Representation gaps
- Contribution language
- AI-generated summaries
- Access permissions
Document:
- Sources and time period covered
- Outcome harvesting and verification process
- Coding framework and AI tools used
- Human-review process
- Known evidence and representation limitations
- Dashboard update frequency and responsible owner
AI Prompt Library
1. Dashboard Blueprint Prompt
Act as an expert in qualitative M&E dashboard design.
I have a reviewed outcome harvesting dataset for [programme], covering [countries], [institutions], [period] and [stakeholder groups].
The dashboard users are [users]. Their decisions include [decisions].
Propose:
1. Six priority questions the dashboard should answer
2. The fields required for each question
3. A suitable visual or narrative view
4. Necessary filters and drill-downs
5. A limitation or interpretation warning for each view
6. A five-page dashboard structure
Do not recommend a visual unless it supports a stated question.
2. Dataset Quality Review Prompt
Review the supplied outcome harvesting table for dashboard readiness.
Check for:
– Missing identifiers or source references
– Duplicate or near-duplicate outcomes
– Inconsistent category values
– Outcomes that do not describe a specific change
– Unsupported significance or contribution claims
– Missing validation status
– Sensitive or potentially identifying information
– Fields needed for stakeholder representation analysis
Return:
Issue | Outcome ID | Why it matters | Recommended correction
Do not rewrite or delete any outcome automatically.
3. Missing-Voice Analysis Prompt
Compare the programme stakeholder map with the outcome harvesting dataset.
For each stakeholder group, identify:
– Whether the group was expected to be represented
– Whether members were consulted directly
– Number of outcome narratives involving the group
– Number based on the group’s direct voice
– Number based only on intermediary accounts
– Representation status: adequate, limited, missing or unknown
– Possible consequence for interpretation
– Recommended additional evidence collection
Do not assume that an absent perspective means no change occurred.
4. Dashboard Narrative Prompt
Write a concise dashboard narrative based only on the filtered outcome records.
Use this structure:
1. What is visible in the records
2. Which Outcome IDs support the observation
3. Important differences by country, institution or stakeholder
4. Evidence quality and validation status
5. One limitation affecting interpretation
6. One question requiring human follow-up
Do not use words such as caused, proved, successful or representative unless the supplied evidence explicitly supports them.
Practical Exercise
Scenario
A fictional regional programme supports gender-responsive institutional change in three countries. Activities include leadership training, institutional dialogue, mentoring, policy advice and technical assistance. The outcome harvesting process produced 48 reviewed outcomes from interviews, meeting minutes, event reports and policy documents.
| ID | Country | Institution | Activity | Outcome level | Stakeholder | Outcome statement | Status |
|---|---|---|---|---|---|---|---|
| OH-01 | A | Training academy | Training | Individual | Women officers | Participants began raising gender-related operational concerns in planning meetings. | Verified |
| OH-02 | A | Regional directorate | Dialogue | Institutional | Managers | The directorate introduced a standing gender agenda item in quarterly reviews. | Verified |
| OH-03 | B | Training academy | Technical assistance | Institutional | Training staff | The academy revised two course modules to include gender-responsive leadership scenarios. | Verified |
| OH-04 | B | Local unit | Mentoring | Relational | Junior staff | Junior women staff reported more frequent access to senior decision-makers. | Partially verified |
| OH-05 | C | Ministry department | Policy advice | Policy | Policy staff | A draft recruitment policy added a requirement to review gender barriers. | Unverified |
| OH-06 | C | Local unit | Training | Individual | Male supervisors | Several supervisors reported greater confidence in addressing discriminatory comments. | Partially verified |
Your Task
- Identify the minimum additional fields required for dashboard analysis.
- Draft three question-first dashboard views.
- Create an outcome-level overview.
- Design an activity-to-change matrix.
- Identify which stakeholder perspectives cannot be assessed from the sample.
- Write one evidence-grounded summary without making a causal claim.
- List two follow-up verification questions.
- Identify information that should be protected in a public dashboard.
Common Interpretation Errors
| Error | Why it is misleading | Better practice |
|---|---|---|
| More narratives mean more change. | Documentation and sampling intensity may differ. | Display evidence volume and collection coverage. |
| Frequent outcomes are the most important. | Frequency does not equal significance. | Combine counts with significance criteria and narratives. |
| An activity caused every linked outcome. | Co-occurrence does not demonstrate attribution. | Review contribution evidence and alternative factors. |
| A missing group experienced no change. | The group may not have been consulted. | Report the gap and collect additional evidence. |
| AI summaries are neutral. | They reflect the data, codebook and prompt. | Require source references, limitations and human review. |
| Charts can replace qualitative interpretation. | Counts remove context and meaning. | Keep drill-down access to narratives and evidence. |
Frequently Asked Questions
Can I build the dashboard directly from interview transcripts?
Not responsibly. First extract, formulate, consolidate and review the outcomes. The dashboard should use a controlled outcome dataset, while authorised users may retain access to the underlying evidence.
Should I count every mention of an outcome?
Usually no. Several mentions may refer to one distinct change. Preserve the number and diversity of supporting sources separately from the number of outcomes.
Can the dashboard show programme attribution?
Outcome harvesting generally examines contribution. The dashboard can display contribution evidence, alternative factors and verification status, but it should not convert association into causal attribution.
Which tool is best?
The best tool is the one your team can govern, maintain and use. Start with the questions, dataset and access requirements. A well-designed spreadsheet prototype is better than an advanced platform that the organisation cannot sustain.
How often should the dashboard be updated?
Update it when new outcomes have completed the agreed review and validation process. Real-time refresh is rarely appropriate for evidence that requires qualitative coding and human judgement.
Final Checklist
- Outcome statements have been reviewed and deduplicated.
- Every outcome has a unique ID and source reference.
- The data dictionary defines all categories.
- Validation status and evidence confidence are visible.
- Contribution is separated from attribution.
- Stakeholder voice and representation fields are included.
- Dashboard questions are linked to real decisions.
- Counts are presented with appropriate limitations.
- Every visual links back to narratives and evidence.
- AI summaries cite supporting Outcome IDs.
- Contradictions and evidence gaps remain visible.
- Sensitive data are protected through appropriate access controls.
- Intended users have reviewed the dashboard.
- The methodology and limitations are documented.
- An owner and update process have been assigned.
Key Takeaway
An effective outcome harvesting dashboard does not reduce qualitative evidence to a collection of colourful counts. It makes patterns easier to explore while preserving context, traceability, uncertainty and stakeholder voice. AI can support this work by structuring records, identifying evidence gaps and drafting summaries, but the quality of the dashboard depends on the judgement, validation and ethical safeguards applied by the M&E team.
Related Resource
Need to identify, extract, consolidate and verify outcomes before building the dashboard? Use the complete AI outcome harvesting workflow.
