AI for Random Sampling and Propensity Score Matching
EvalCommunity Practical Tutorial
How to Use AI for Random Sampling and Propensity Score Matching in M&E
Learn how AI tools such as Claude, ChatGPT and Julius AI can assist with sampling-frame inspection, reproducible random sampling, propensity-score modelling, matching, diagnostics and statistical documentation.
Random & stratified
Propensity scores
Claude, ChatGPT & others
Prompts + workflows
The Short Answer
Yes — AI can assist with both.
AI can help evaluators inspect sampling frames, design sampling procedures, generate reproducible code, estimate propensity scores, perform matching workflows and interpret diagnostics. The actual statistical design, assumptions and final interpretation remain the responsibility of the evaluator.
First: Do Not Confuse These Two Problems
Random sampling and propensity-score matching are not the same thing. They answer different evaluation questions.
Remember:
Sampling determines how observations are selected. Propensity-score methods address comparability in observational treatment groups.
◉
What You Will Learn
01. Sampling
Use AI to inspect and design sampling procedures.
02. Random Selection
Generate reproducible computational sampling code.
03. Propensity Scores
Understand and estimate treatment probabilities.
04. Matching
Use AI alongside R, MatchIt, Stata or SPSS.
05. Diagnostics
Check overlap and covariate balance.
06. QA
Know where human methodological review is required.
Which Method Are You Actually Looking For?
↓
Do you need to select observations from a defined population?
If yes, you are dealing with a sampling problem.
Are you comparing treated and untreated observations in observational data?
If yes, you may need a causal-inference strategy.
The AI-Assisted M&E Workflow
Inspect Data
Define Method
Generate Code
Run Analysis
Check Diagnostics
Human Review
AI assists → statistical software computes → evaluator verifies → evaluator interprets
1. Can AI Perform Random Sampling?
Yes. AI can help design and implement a reproducible sampling procedure. But an AI language model should not simply invent a list of “random” beneficiary IDs.
Key principle:
Use AI to inspect, design, code and quality-check the sampling process. Use an actual computational random-number procedure against the real sampling frame to make the selection.
Example: 500 beneficiaries from 8,000 records
Imagine an evaluation team has a beneficiary database containing 8,000 records and needs a sample of 500 people.
| Variable | Example |
|---|---|
| beneficiary_id | B00001 |
| region | North |
| district | District A |
| sex | Female |
| age | 34 |
| programme_status | Participant |
| baseline_score | 62 |
Prompt: Inspect the Sampling Frame
You are supporting an M&E evaluation team.
We need to select a probability sample from the attached beneficiary sampling frame.
The target sample size is 500.
Before selecting anything:
1. Inspect the dataset structure.
2. Identify the number of records.
3. Check whether beneficiary IDs are unique.
4. Identify duplicate records.
5. Identify missing beneficiary IDs.
6. Check for obvious data-quality problems.
7. Identify variables that could be used for stratification.
8. Explain whether simple random sampling or stratified random sampling appears appropriate based on the information available.
Do not select the sample yet.
Do not invent or modify any records.
Clearly distinguish observations about the dataset from methodological recommendations.
2. Simple Random Sampling with AI
If simple random sampling is appropriate, AI can generate reproducible code. The procedure should operate on the actual sampling frame and document a random seed.
set.seed(20260813)
selected <- data[
sample(
seq_len(nrow(data)),
size = 500,
replace = FALSE
),
]
Why the seed matters:
A documented seed allows the same procedure to be reproduced when the same data and software environment are used.
3. Stratified Random Sampling
For many M&E evaluations, simple random sampling is not necessarily the best design. If the population contains meaningful subgroups, stratification may be appropriate.
8,000
Stratum 1
Stratum 2
Additional strata
AI can help with:
- calculating population sizes by stratum;
- calculating sample allocations;
- identifying small or missing strata;
- generating reproducible sampling code;
- checking the final allocation.
Prompt for Stratified Sampling
Using the attached beneficiary sampling frame:
I need a total probability sample of 500 beneficiaries.
Assess whether proportional stratified random sampling by region is appropriate.
First calculate:
– total population size;
– population size in each region;
– proportion of the population represented by each region;
– proportional sample allocation for a total sample of 500.
Then provide reproducible R code that:
1. uses region as the stratum;
2. selects the required number of beneficiaries within each region;
3. samples without replacement;
4. uses a documented random seed;
5. does not alter the original dataset.
Before providing the code, explain any assumptions or issues you identify.
Do not invent values that are not present in the dataset.
Try It Yourself
Exercise 1: Select a Sample of 100
Imagine you have a fictional beneficiary dataset containing 2,000 records. Your task is to select a probability sample of 100.
Your task:
- Ask Claude or ChatGPT to inspect the sampling frame.
- Determine whether simple or stratified sampling appears appropriate.
- Generate reproducible R code.
- Run the code.
- Verify the final sample.
4. What Is a Propensity Score?
Propensity-score analysis is different from random sampling. It is primarily used in observational studies where treatment or programme participation was not randomly assigned.
A propensity score represents the estimated probability of receiving treatment given specified observed characteristics.
Concept
P(Treatment = 1 | pre-treatment covariates)
The Propensity-Score Workflow
Who participated?
Pre-treatment factors
Treatment probability
Comparable observations
Balance & overlap
5. Can Claude Help Generate Propensity Scores?
Yes. Claude can assist with data analysis and coding where the relevant capabilities are available. It can help inspect data, generate statistical code, explain models and interpret diagnostic output.
Important:
The ability to generate or run an analysis does not mean that the selected causal design is valid. The evaluator still needs to define the question, treatment, outcome, covariates, estimand and assumptions.
6. Start With the Causal Question
Do not begin by asking an AI tool to “run propensity-score matching.” Begin with the causal question.
Example:
What is the effect of participating in a vocational training programme on employment six months after training?
Treatment
Programme participation
Outcome
Employment six months later
Covariates
Pre-treatment characteristics
7. Ask AI to Review Variables Before Modelling
AI can be particularly useful for reviewing a variable dictionary and flagging variables whose timing or role is unclear.
I am preparing a propensity-score analysis for an M&E evaluation.
Treatment:
programme participation
Outcome:
employment status six months after the intervention
For each variable in the attached dataset, classify it as:
A. Clearly pre-treatment
B. Potentially pre-treatment but requires clarification
C. Post-treatment
D. Outcome variable
E. Identifier
F. Administrative/technical variable
G. Unclear
Explain the reasoning for every variable classified as B, C or G.
Do not decide the final propensity-score specification for me.
Do not infer measurement timing when it is not documented.
Flag anything that requires confirmation from the evaluation team.
8. Using R and MatchIt
For evaluators working in R, MatchIt provides a framework for matching in observational studies. It supports multiple matching approaches and provides tools for assessing covariate balance and overlap.
library(MatchIt)
m.out <- matchit(
treatment ~ age +
education +
baseline_income +
baseline_employment +
baseline_score +
region,
data = data,
method = “nearest”,
distance = “glm”,
link = “logit”,
ratio = 1
)
summary(m.out)
matched_data <- match.data(m.out)
Do not copy this specification blindly.
The treatment model, covariates, matching method, estimand and diagnostics should reflect the actual evaluation design.
9. What Happens After Matching?
Before
Compare treated and control groups.
Overlap
Check common support.
Balance
Assess covariate balance.
Interpret
Only after diagnostics.
10. Check Common Support
Common support refers to overlap in the propensity-score distributions of treated and comparison observations. If the groups occupy very different parts of the distribution, suitable matches may not exist for many observations.
Ask AI:
Examine the propensity-score distribution separately for treated and control observations. Assess whether there appears to be adequate overlap/common support. Identify regions where matching may be problematic. Do not simply state that common support exists; explain the evidence.
11. Check Covariate Balance
The purpose of matching is not to create attractive-looking propensity scores. It is to improve comparability on observed covariates under the assumptions of the design.
Compare covariate differences.
Inspect propensity-score overlap.
Check retained and discarded cases.
Interpret this propensity-score matching balance table as a statistical reviewer.
Focus on:
1. Whether balance improved after matching.
2. Which covariates remain poorly balanced.
3. Whether the propensity-score distribution suggests adequate overlap.
4. How many observations were lost through matching.
5. Whether the matching specification appears adequate based on these diagnostics.
Do not claim that matching establishes causal identification.
Distinguish clearly between:
– what the diagnostics show;
– what they do not show;
– what requires further methodological judgment.
Practical Lab
Exercise 2: Build a Propensity-Score Workflow
You have an observational dataset containing programme participants and eligible non-participants.
Your task:
- Identify the treatment variable.
- Identify the outcome.
- Classify candidate pre-treatment covariates.
- Ask Claude to flag timing problems.
- Generate a MatchIt workflow.
- Check common support.
- Assess covariate balance.
- Write a cautious interpretation.
Using AI: Do This, Not That
12. What About Stata and SPSS?
AI can also help evaluators who work in Stata or SPSS by generating syntax, explaining commands, troubleshooting errors and helping interpret statistical output.
teffects psmatch (outcome) ///
(treatment age education baseline_income baseline_score), ///
atet
The exact syntax and estimation method should always be checked against the version of the statistical software and the evaluation design.
13. Which AI Tool Should You Use?
| Tool | Useful For | Best Used With |
|---|---|---|
| Claude | Reasoning, coding, data review and documentation | R, MatchIt, Stata or SPSS |
| ChatGPT | Data analysis, charts, calculations and coding | Statistical software for verification |
| Julius AI | Conversational data analysis and visualization | Human statistical review |
| R + MatchIt | Matching and diagnostics | Evaluator + AI coding assistance |
| Stata | Causal inference and treatment effects | AI for syntax and QA |
| SPSS | Statistical analysis and workflows | AI for syntax and interpretation |
14. One Master Prompt for Claude
You are assisting an M&E evaluation team with an observational impact analysis.
Your role is to act as a statistical programming and quality-control assistant, not as the final methodological decision-maker.
Evaluation question:
[INSERT EVALUATION QUESTION]
Treatment/intervention:
[INSERT TREATMENT]
Outcome:
[INSERT OUTCOME]
Treatment timing:
[INSERT WHEN TREATMENT OCCURRED]
Dataset:
[DESCRIBE OR ATTACH DATASET]
Our confirmed pre-treatment covariates are:
[INSERT COVARIATES]
Target estimand:
[ATE / ATT / ATC]
Requested method:
[INSERT METHOD]
Your workflow:
1. Inspect the dataset without changing it.
2. Report the number of observations and variables.
3. Check identifiers and duplicates.
4. Check missingness.
5. Check treatment coding.
6. Check outcome coding.
7. Check candidate covariates.
8. Flag variables whose timing is unclear.
9. Do not introduce post-treatment variables into the treatment model.
10. Estimate propensity scores using the specified method.
11. Show the model specification.
12. Examine common support/overlap.
13. Perform the specified matching procedure.
14. Report observations retained and discarded.
15. Assess covariate balance before and after matching.
16. Produce appropriate diagnostic tables and plots.
17. If balance is poor, explain the problem rather than silently changing the model.
18. Provide reproducible code.
19. Clearly separate calculations from methodological recommendations.
20. Do not invent statistical results.
21. Do not invent missing data.
22. Do not claim causal identification merely because matching was performed.
For every important conclusion, identify the evidence supporting it.
Before the final interpretation, provide a section called:
“Human methodological review required”
List the decisions that still require review by an evaluator or statistician.
15. AI Prompt Library for Sampling & Matching
Prompt 01
Audit a sampling frame.
Prompt 02
Assess possible stratification.
Prompt 03
Generate reproducible sampling code.
Prompt 04
Review propensity-score variables.
Prompt 05
Generate MatchIt code.
Prompt 06
Review common support.
Prompt 07
Interpret balance diagnostics.
Prompt 08
Conduct methodological QA.
Human Methodological Review Required
AI Can Assist
Data inspection, coding, calculations, diagnostics and documentation.
Evaluator Must Own
Sampling design, causal assumptions, estimand, covariate decisions and interpretation.
16. Practical M&E Example
An NGO provides entrepreneurship training to 1,200 people. Participation was voluntary. Six months later, the evaluation team measures whether participants have an active income-generating activity.
Programme participants
Eligible non-participants
Demographics, income, employment
Outcome measurement
Evaluation question:
What was the effect of participating in the entrepreneurship training programme on having an active income-generating activity six months later?
AI can help audit the data, review variable timing, generate the R or Stata implementation, troubleshoot the analysis, interpret diagnostics and document the workflow.
Knowledge Check
Test Yourself
1. What is the primary purpose of random sampling?
A. Estimate propensity scores B. Select observations from a population C. Remove confounding D. Estimate treatment effects
2. What does a propensity score represent?
A. Probability of the outcome B. Probability of treatment given observed covariates C. Sampling fraction D. Treatment effect
3. Which variable should not automatically be included as a pre-treatment covariate?
A. Age measured before treatment B. Baseline income C. Post-treatment outcome D. Baseline employment
4. Why check covariate balance after matching?
A. To determine whether matching improved comparability B. To calculate sample size C. To generate random numbers D. To prove causality
Answer key:
1 — B | 2 — B | 3 — C | 4 — A
17. One-Page Cheat Sheet
RANDOM SAMPLING
Population
↓
Sampling frame
↓
Data-quality check
↓
Sampling design
↓
Reproducible sample
PROPENSITY METHODS
Causal question
↓
Treatment + outcome
↓
Pre-treatment covariates
↓
Propensity score
↓
Matching + balance
↓
Cautious interpretation
Recommended Tutorial Resource Pack
Practice With Realistic M&E Files
CSV practice dataset
Random sampling workflow
PSM Dataset
Matching + diagnostics
Sampling + matching promptsl.
18. Responsible Use of AI
AI does not create random assignment.
Propensity-score matching does not transform observational data into a randomized controlled trial.
Matching does not remove all confounding.
Matching addresses observed covariates under its assumptions. It does not automatically remove unmeasured confounding.
Protect beneficiary data.
Use de-identified data where possible and follow organizational, donor and applicable data-protection requirements.
19. Six Common Mistakes
Asking AI to invent a random sample.
Copying statistical code without running it.
Including post-treatment variables.
Ignoring common support.
Looking only at propensity scores.
Treating AI-generated interpretation as causal evidence.
20. The Five-Step Verification Rule
Read the code
Run the code
Check diagnostics
Reproduce results
Interpret professionally
The Bigger Lesson
Let AI help you perform the analysis — not decide what the analysis means.
The strongest workflow combines AI assistance with established statistical software and human methodological review.
Continue Learning with EvalCommunity Academy
Go deeper into AI-assisted quantitative and causal analysis
This tutorial connects directly with broader EvalCommunity learning on AI-enhanced quantitative analysis, causal inference, impact evaluation, human-in-the-loop quality assurance and AI-powered M&E workflows.
AI in Monitoring & Evaluation
Build practical AI skills across the M&E lifecycle.
LEARN MORE
AI for Causal Inference
Explore counterfactuals, causal inference and impact evaluation.
LEARN MORE
Frequently Asked Questions
Can Claude calculate propensity scores?
Yes. Claude can assist with data analysis and coding where the relevant capabilities are available. Always verify the model, code, assumptions and results.
Can ChatGPT analyse an uploaded dataset?
ChatGPT currently supports uploaded-file data analysis and can create tables and charts where appropriate. The exact capabilities available can depend on the model, plan and workspace configuration.
Is Claude better than R for propensity-score matching?
They serve different purposes. Claude can assist with reasoning, coding and interpretation, while R and packages such as MatchIt provide the statistical implementation.
Can AI randomly select my evaluation sample?
AI can help implement a computational random-sampling procedure. The actual selection should operate against the real sampling frame using a reproducible random-number process.
Can AI choose propensity-score covariates?
AI can help review and classify candidate variables, but the final specification should be based on the evaluation design, causal reasoning and knowledge of when variables were measured.
Does propensity-score matching prove causality?
No. Matching can improve comparability on observed covariates, but it does not automatically eliminate unmeasured confounding or guarantee causal identification.
Final Takeaway
AI can assist the analysis. It should not replace the methodology.
For random sampling, use AI to inspect the frame, design the procedure and generate reproducible code. For propensity scores, use AI to assist with coding, diagnostics and documentation while established statistical tools perform the formal analysis.
