EvalCommunity Academy · Survey Scoping Series

The SCOPE Tutorial

Seven components to operationalise AI-assisted survey scoping while keeping measurement decisions firmly in human hands.

S · Structure
C · Generate Options
O · Overlap Check
P · Probe Assumptions
E · Evaluate

7 Components
Full Construct Mapping
6 AI Prompts
Human Evaluation

C1 Decision Interface
C2 Mapping
C3 Prompts
C4 Reference Card
C5 Checklist
C6 Sample Audit
C7 Audit Log

Component 1

SCOPE Decision Interface

Run an indicator through all five steps. AI assists with generating, organising, flagging, and surfacing assumptions. Evaluation remains human-only.

Constraint in force: no AI-generated question is accepted automatically. Every question must pass manual E-step evaluation.

Indicator Assessment Form

Indicator Name *

Evaluator Name *

Date *

Program / Context

S · Structure

Define the construct before reviewing any AI output.

Your construct definition *

Indicator dimensions

C · Generate Options

Paste AI outputs as raw material only. Do not accept them here.

Prompt used

AI-generated questions *

O · Overlap Check

AI flags redundancy; you decide combine, keep distinct, or drop.

P · Probe Assumptions

Every measurement assumption must be reviewed before a question is retained.

Assumption	Valid?	Revision / Action

E · Evaluate — Human Only

AI stops here. Every decision requires explicit human reasoning linked to the construct definition.

Question wording	What it measures	Maps?	Decision	Justification

Final human sign-off

Component 2

Construct–Question Mapping Template

Every retained question must trace a direct line back to your construct definition.

Unique ID	Construct Facet	Question Wording	Mapping Justification	Decision
SCOPE-001	Agency	Final approved wording	Why it operationalises this facet	KEEP / REVISE / DISCARD

Component 3

AI-Assisted Scoping Prompt Library

Each prompt requests options, never decisions. Copy the prompt text manually from the boxes below.

Prompt S1 — Generate indicator dimensions from literature

I am scoping a survey indicator. I need the theoretical dimensions of this construct from relevant literature.
Indicator / Construct: [YOUR INDICATOR]
Program context: [BRIEF CONTEXT]
Domain: [DOMAIN]
Instructions:
1. List 5-8 theoretical dimensions from published literature.
2. For each, provide a one-sentence definition and one example measurement approach.
3. Note contested dimensions.
4. Do not suggest which to prioritise.
5. End with: "RAW MATERIAL — Evaluator must define which dimensions apply to their specific context."

Prompt C1 — Generate 5-7 question options

I am scoping a survey for monitoring and evaluation. I need multiple question options — not a single best question.
Indicator: [YOUR INDICATOR]
Construct definition: [YOUR DEFINITION — written before this prompt]
Context: [PROGRAM, COUNTRY/REGION, TARGET POPULATION]
Survey mode: [MODE]
Instructions:
1. Generate exactly 5-7 question options.
2. Include direct, indirect, behavioral, attitudinal, and frequency-based options.
3. Use neutral language.
4. Do not recommend which to use.
5. End with: "SCOPE C-step output: These are raw material options. Evaluator must apply Step E before any question is retained."

Prompt O1 — Identify overlap / redundancy

I have draft survey questions for one indicator. Identify potential redundancy before construct validity evaluation.
Indicator: [YOUR INDICATOR]
Draft questions: [PASTE ALL QUESTIONS]
Instructions:
1. Group questions measuring the same underlying concept.
2. Explain why they might be redundant.
3. Do not tell me which to drop.
4. Note questions measuring different facets.
5. End with: "SCOPE O-step output: These are flagged overlaps only. Evaluator must decide."

Prompt P1 — Surface hidden assumptions

Audit the assumptions embedded in these draft survey questions before deciding which to retain.
Indicator: [YOUR INDICATOR]
Questions: [PASTE QUESTIONS]
For each question, list assumptions about literacy, technology access, culture, recall, terminology, response bias, and power dynamics.
Format: [Question] | [Assumption] | [Category]
Do not evaluate whether assumptions are valid — that is my judgment.

Component 4

SCOPE Reference Card

AI generates options; you evaluate and decide. The E in SCOPE is non-delegable.

S — Structure: define construct before AI.
C — Generate options: AI drafts options only.
O — Overlap check: AI flags redundancy; you decide.
P — Probe assumptions: AI surfaces assumptions; you validate.
E — Evaluate: human-only judgment.

Component 5

Assumption Audit Checklist

Respondent can understand the question as written.

Respondent has access to any devices or services referenced.

Individual decision-making is appropriate in context.

Topic is safe to discuss with an interviewer.

Recall period is realistic.

Key terms translate clearly into the local language.

Component 6

Sample Completed SCOPE Audit

Indicator: Women’s Economic Empowerment

Construct definition: Women’s ability to control economic resources and make independent financial decisions.

Question	Decision	Reason
Who usually makes the final decision about household money?	Keep	Directly measures agency over financial decisions.
Do you have a bank account in your name?	Revise	Formal banking access assumption may not hold; include mobile money and savings groups.
Do you feel confident managing finances?	Discard	Measures self-efficacy, not empowerment/control.

Component 7

SCOPE Audit Log

Use this table to track scoped indicators across a programme or portfolio.

Audit ID	Date	Indicator	Generated	Retained	% Revised/Discarded	Evaluator
SCOPE-001

SCOPE Toolkit · EvalCommunity Academy · AI-assisted survey scoping with human-only evaluation decisions.

SCOPE Toolkit Tutorial