PSM Dataset: Use Fictional Evaluation Data for Propensity Score Matching in R

Propensity Score Matching (PSM) is a method for analysing observational evaluation data when treatment and comparison groups differ before an intervention.

In this tutorial, you will work with a fictional M&E dataset and use R to estimate propensity scores, match treated and comparison participants, assess covariate balance, and estimate a simple outcome difference after matching.

Important: The dataset and results in this tutorial are fictional and are intended for learning. They should not be interpreted as evidence from a real programme.

What You Will Learn

By the end of this tutorial, you will be able to:

Understand the structure of a PSM dataset.
Identify treatment, outcome and baseline covariates.
Prepare fictional evaluation data in R.
Estimate propensity scores.
Perform nearest-neighbour propensity score matching.
Assess whether matching improved covariate balance.
Extract the matched dataset.
Estimate a simple outcome difference after matching.
Interpret the results cautiously.
Identify common PSM mistakes in M&E.

What You Need

R or RStudio.
The fictional evaluation dataset created in this tutorial.
The MatchIt R package.
Basic familiarity with R data frames and syntax.

MatchIt documentation

What Is Propensity Score Matching?

A propensity score is the probability that a unit receives a treatment or intervention given a set of observed covariates.

For example, suppose an NGO evaluates a livelihood training programme. Participation was not randomly assigned. People who joined the training may differ from non-participants in:

Age
Education
Household size
Baseline income
Previous employment
Distance from the training centre

These characteristics may also be related to the evaluation outcome.

PSM attempts to make the treatment and comparison groups more comparable with respect to the observed baseline characteristics.

A typical PSM workflow is:

Estimate propensity scores
        ↓
Match treatment and comparison observations
        ↓
Check covariate balance
        ↓
Check overlap
        ↓
Analyse the matched outcome
        ↓
Interpret assumptions and limitations

PSM is not a replacement for randomization. It does not automatically eliminate unmeasured confounding.

A Fictional M&E Evaluation

Imagine an NGO implemented a six-month livelihood training programme.

The evaluation team wants to examine whether participation in the programme is associated with higher household income at endline.

There was no randomized assignment. Instead, people chose whether to participate in the training.

The evaluation dataset contains:

id — participant identifier
treatment — whether the participant received the training
age — participant age
education — years of education
household_size — household size
baseline_income — income measured before the intervention
distance_km — distance from the training centre
endline_income — income measured after the intervention

The variables used to estimate the propensity score should generally be measured before treatment or otherwise be appropriate pre-treatment covariates.

Step 1: Install the Required R Packages

Install MatchIt:

install.packages("MatchIt")

Load it:

library(MatchIt)

You can also use dplyr for data manipulation:

install.packages("dplyr")
library(dplyr)

You only need to install a package once.

Step 2: Create a Fictional PSM Dataset

For this tutorial, you can generate a fictional dataset directly in R.

The following code creates 500 fictional participants:

set.seed(2026)
n <- 500
psm_data <- data.frame(
  id = 1:n,
  age = round(rnorm(n, mean = 38, sd = 10)),
  education = pmax(0, round(rnorm(n, mean = 9, sd = 3))),
  household_size = sample(2:8, n, replace = TRUE),
  baseline_income = round(
    rlnorm(n, meanlog = 6.5, sdlog = 0.5)
  ),
  distance_km = round(
    runif(n, 0.5, 20),
    1
  )
)

At this stage, you have the baseline characteristics but not yet the treatment or outcome variables.

Step 3: Generate a Fictional Treatment Variable

For this fictional example, treatment assignment is generated from the baseline characteristics.

This deliberately creates some differences between the treatment and comparison groups so that the PSM workflow has something to address.

psm_data$treatment_probability <- plogis(
  -2.2 +
  0.025 * psm_data$age +
  0.10 * psm_data$education +
  0.06 * psm_data$household_size -
  0.0004 * psm_data$baseline_income -
  0.04 * psm_data$distance_km
)
psm_data$treatment <- rbinom(
  n,
  size = 1,
  prob = psm_data$treatment_probability
)

The variable treatment contains:

0 = comparison group
1 = treatment group

Check the number of participants in each group:

table(psm_data$treatment)

You can also calculate the proportions:

prop.table(table(psm_data$treatment))

Because this is simulated data, the exact numbers will vary if you change the random seed.

Step 4: Generate a Fictional Outcome

Now create a fictional endline income variable.

psm_data$endline_income <- pmax(
  0,
  round(
    psm_data$baseline_income +
    1500 * psm_data$treatment +
    100 * psm_data$education +
    rnorm(n, 0, 1200)
  )
)

The pmax() function prevents the simulated income variable from becoming negative.

Important: The generated treatment effect is fictional. It should not be interpreted as evidence that the programme actually increases income by a particular amount.

Step 5: Inspect the Dataset

Look at the first observations:

head(psm_data)

Check the structure:

str(psm_data)

Check the treatment groups:

table(psm_data$treatment)

Check the summary statistics:

summary(psm_data)

Compare baseline income by treatment group:

aggregate(
  baseline_income ~ treatment,
  data = psm_data,
  FUN = mean
)

This helps illustrate why matching may be useful when treatment and comparison groups differ in their baseline characteristics.

Step 6: Identify the Variables for the Propensity Model

For this fictional evaluation, we will use:

Age
Education
Household size
Baseline income
Distance from the training centre

The treatment variable is:

treatment

The outcome is:

endline_income

The outcome should not be used to determine who gets matched.

The propensity model should be based on appropriate pre-treatment variables and substantive knowledge about treatment assignment and potential confounding.

Step 7: Perform 1:1 Nearest-Neighbour Matching

Now use MatchIt to perform nearest-neighbour matching.

match_model <- matchit(
  treatment ~ age +
    education +
    household_size +
    baseline_income +
    distance_km,
  data = psm_data,
  method = "nearest",
  distance = "glm",
  ratio = 1,
  replace = FALSE
)

This specifies:

treatment as the treatment indicator.
The other variables as baseline covariates.
method = "nearest" for nearest-neighbour matching.
distance = "glm" to estimate the propensity score using a generalized linear model.
ratio = 1 for one-to-one matching.
replace = FALSE so comparison observations are not reused.

The matching specification should be chosen according to the evaluation design and causal estimand rather than simply copied from an example.

Step 8: Examine the Matching Results

Start with:

summary(match_model)

The summary provides information about the sample before and after matching and the balance of the covariates.

Do not assume that matching automatically makes the groups comparable. You need to assess whether matching actually improved balance.

Step 9: Check Covariate Balance

A key purpose of matching is to make the treatment and comparison groups more similar on observed baseline covariates.

summary(
  match_model,
  standardize = TRUE
)

Examine the standardized mean differences before and after matching.

The exact threshold considered acceptable should be justified according to the evaluation context rather than treated as a universal rule.

The key question is:

Did the matching procedure materially improve balance on the observed baseline covariates?

Step 10: Extract the Matched Dataset

Once you have inspected the matching results, extract the matched data:

matched_data <- match.data(match_model)

Inspect the data:

head(matched_data)

Check the treatment distribution:

table(matched_data$treatment)

Check the number of observations:

nrow(matched_data)

Inspect the matching weights:

summary(matched_data$weights)

Step 11: Examine the Matched Groups

Compare the baseline variables again.

Age:

aggregate(
  age ~ treatment,
  data = matched_data,
  FUN = mean
)

Education:

aggregate(
  education ~ treatment,
  data = matched_data,
  FUN = mean
)

Baseline income:

aggregate(
  baseline_income ~ treatment,
  data = matched_data,
  FUN = mean
)

The goal is not simply to make the sample sizes equal. The goal is to improve comparability of the observed baseline characteristics.

Step 12: Estimate a Simple Outcome Difference

For this fictional example, calculate mean endline income by treatment group:

aggregate(
  endline_income ~ treatment,
  data = matched_data,
  FUN = mean
)

You can calculate the difference:

mean_treated <- mean(
  matched_data$endline_income[
    matched_data$treatment == 1
  ]
)
mean_control <- mean(
  matched_data$endline_income[
    matched_data$treatment == 0
  ]
)
difference <- mean_treated - mean_control
difference

This is a simple descriptive difference between the matched treatment and comparison observations.

Do not automatically describe this value as a causal effect. A formal impact estimate requires consideration of the estimand, matching design, weighting, uncertainty and causal assumptions.

Step 13: Examine Matching Weights

The matched dataset contains weights generated by MatchIt.

head(matched_data$weights)

For example, weighted means can be calculated as follows:

weighted.mean(
  matched_data$endline_income[
    matched_data$treatment == 1
  ],
  matched_data$weights[
    matched_data$treatment == 1
  ]
)

And for the comparison group:

weighted.mean(
  matched_data$endline_income[
    matched_data$treatment == 0
  ],
  matched_data$weights[
    matched_data$treatment == 0
  ]
)

The appropriate outcome analysis depends on the matching design and estimand. A simple weighted mean is not automatically the final treatment-effect analysis.

Step 14: Examine Propensity Score Overlap

Overlap is important because matching requires comparable treated and comparison observations.

Inspect the propensity scores:

plot(
  match_model,
  type = "jitter",
  interactive = FALSE
)

You can also examine the distribution using:

plot(
  match_model,
  type = "hist"
)

The purpose is to examine whether treated and comparison observations have reasonable overlap in their estimated propensity scores.

Limited overlap can make matching difficult and can affect the population to which the analysis applies.

Step 15: Use a Caliper

A caliper restricts how far apart matched observations can be on the matching distance.

For example, the following uses a standardized caliper:

match_model_caliper <- matchit(
  treatment ~ age +
    education +
    household_size +
    baseline_income +
    distance_km,
  data = psm_data,
  method = "nearest",
  distance = "glm",
  ratio = 1,
  replace = FALSE,
  caliper = 0.2,
  std.caliper = TRUE
)

Here, std.caliper = TRUE makes the caliper value a standardized distance rather than a raw distance on the propensity-score scale.

After changing the matching specification, repeat the balance assessment:

summary(
  match_model_caliper,
  standardize = TRUE
)

A caliper should be justified as part of the analysis plan rather than selected simply because it produces a preferred result.

A Complete Fictional PSM Workflow

The following script creates the fictional dataset, performs matching and extracts the matched observations.

# -----------------------------------------
# FICTIONAL PSM WORKFLOW FOR M&E
# -----------------------------------------
library(MatchIt)
library(dplyr)
# 1. Set seed for reproducibility
set.seed(2026)
# 2. Create fictional baseline data
n <- 500
psm_data <- data.frame(
  id = 1:n,
  age = round(rnorm(n, 38, 10)),
  education = pmax(
    0,
    round(rnorm(n, 9, 3))
  ),
  household_size = sample(
    2:8,
    n,
    replace = TRUE
  ),
  baseline_income = round(
    rlnorm(
      n,
      meanlog = 6.5,
      sdlog = 0.5
    )
  ),
  distance_km = round(
    runif(n, 0.5, 20),
    1
  )
)
# 3. Generate fictional treatment probabilities
psm_data$treatment_probability <- plogis(
  -2.2 +
  0.025 * psm_data$age +
  0.10 * psm_data$education +
  0.06 * psm_data$household_size -
  0.0004 * psm_data$baseline_income -
  0.04 * psm_data$distance_km
)
# 4. Generate fictional treatment assignment
psm_data$treatment <- rbinom(
  n,
  size = 1,
  prob = psm_data$treatment_probability
)
# 5. Generate fictional endline outcome
psm_data$endline_income <- pmax(
  0,
  round(
    psm_data$baseline_income +
    1500 * psm_data$treatment +
    100 * psm_data$education +
    rnorm(n, 0, 1200)
  )
)
# 6. Estimate propensity scores and match
match_model <- matchit(
  treatment ~ age +
    education +
    household_size +
    baseline_income +
    distance_km,
  data = psm_data,
  method = "nearest",
  distance = "glm",
  ratio = 1,
  replace = FALSE
)
# 7. Examine balance
summary(
  match_model,
  standardize = TRUE
)
# 8. Extract matched data
matched_data <- match.data(match_model)
# 9. Check matched sample
table(matched_data$treatment)
# 10. Compare outcome means
aggregate(
  endline_income ~ treatment,
  data = matched_data,
  FUN = mean
)
# 11. Save matched dataset
write.csv(
  matched_data,
  "fictional_psm_matched_data.csv",
  row.names = FALSE
)

Practical M&E Example

Suppose a livelihoods programme was implemented in 20 communities.

Participation was voluntary, and the evaluation team wants to compare households that participated in training with similar households that did not.

The evaluation team has baseline information on:

Household income
Household size
Education
Distance from the training location
Age of the household head

The team uses these baseline variables to estimate the probability of participating in the programme.

PSM then creates a matched comparison group with similar observed characteristics.

The evaluation team checks whether the standardized differences in the baseline covariates decreased after matching.

Only after assessing balance does the team proceed to analyse the endline outcome.

The sequence matters:

Baseline covariates
        ↓
Propensity score
        ↓
Matching
        ↓
Balance assessment
        ↓
Outcome analysis

How to Verify the Result

Before interpreting a PSM analysis, check the following.

1. Were covariates measured before treatment?

Avoid using variables that could have been affected by the intervention.

2. Was treatment clearly defined?

The treatment variable should clearly distinguish participants from the comparison group.

3. Was the propensity model specified using substantive knowledge?

Do not choose variables simply because they produce the most attractive matching result.

4. Did matching improve balance?

Compare balance before and after matching.

5. Is there sufficient overlap?

Check whether treated and comparison observations have comparable propensity scores.

6. How many observations were discarded?

Matching can exclude observations that cannot be adequately matched. Document how many observations were removed and why.

7. Which estimand is being estimated?

For example, are you targeting the:

Average Treatment Effect (ATE)?
Average Treatment Effect on the Treated (ATT)?
Another population defined by the matching procedure?

8. Was uncertainty estimated appropriately?

A point estimate alone is not enough for a complete impact evaluation analysis.

Common Mistakes to Avoid

Matching on post-treatment variables

Do not automatically include variables that may have been changed by the intervention.

Matching only because treatment groups look different

PSM should be based on a defensible causal framework and knowledge of potential confounders, not simply a search for variables with statistically significant baseline differences.

Checking only the propensity score

Good overlap in propensity scores does not necessarily mean that every important covariate is adequately balanced. Check covariate balance directly.

Assuming PSM removes all bias

PSM can reduce imbalance associated with observed covariates under the relevant assumptions. It cannot directly balance important unobserved confounders that are absent from the analysis.

Using the outcome to select matches

The outcome should not be used to manipulate the matching specification simply to produce a preferred treatment estimate.

Ignoring observations that were dropped

If matching removes many participants, investigate why. The resulting estimate may apply to a different population than the original evaluation sample.

Treating matched data as randomized data

Matching can improve comparability, but it does not recreate the randomization process of an actual randomized controlled trial.

Limitations of PSM in M&E

PSM is useful, but it has important limitations.

First, it depends on the quality of the observed baseline covariates. If an important confounder was not measured, matching cannot directly balance it.

Second, poor overlap between treatment and comparison groups can make matching difficult.

Third, different propensity-score specifications and matching choices can produce different matched samples.

Fourth, matching can reduce the number of observations available for analysis.

Finally, interpretation depends on the estimand, matching design and underlying assumptions.

For observational evaluation data, PSM should therefore be treated as part of a broader causal-inference strategy rather than as a mechanical procedure that automatically produces an unbiased impact estimate.

Using AI to Assist With PSM

AI can be useful when working with R and PSM, particularly for explaining code, debugging syntax and reviewing whether a script is doing what you intended.

For example, you could use the following prompt:

You are supporting an M&E evaluator working with observational
evaluation data.
Review the following R propensity score matching code.
Identify:
1. Coding errors
2. Variables used in the propensity model
3. Whether any variables appear to be post-treatment variables
4. The matching method being used
5. The estimand being targeted
6. How balance is assessed
7. Potential overlap problems
8. Important assumptions that should be checked
Do not invent information that is not present in the dataset or code.
Do not claim that matching establishes causality.
Explain which issues require methodological judgment by the evaluator.

AI can help inspect the workflow, but the evaluator remains responsible for the causal model, covariate selection, matching specification and interpretation.

Frequently Asked Questions

What is a PSM dataset?

A PSM dataset is typically observational data containing a treatment indicator, an outcome and relevant pre-treatment covariates that can be used to estimate treatment probabilities and create comparable treatment and comparison groups.

What variables should be included in propensity score matching?

Variables should generally be selected based on substantive knowledge about factors related to treatment assignment and the outcome, particularly relevant pre-treatment confounders. They should not be selected solely because they produce a preferred result.

What is the treatment variable in PSM?

The treatment variable indicates whether each observation received the intervention or exposure being evaluated. In a simple binary PSM analysis, it is commonly coded as 1 for treated and 0 for comparison.

What does nearest-neighbour matching do?

Nearest-neighbour matching pairs treated observations with comparison observations that have similar values of the matching distance, which is often the estimated propensity score.

Why is balance checking important after PSM?

The purpose of matching is to create more comparable groups on observed covariates. Balance checking determines whether the chosen matching specification actually improved comparability.

Does PSM eliminate selection bias?

No. PSM can reduce imbalance associated with observed covariates under the relevant assumptions, but it cannot directly address important unobserved confounders that are not included in the analysis.

Can PSM be used for impact evaluation?

Yes. PSM can be used as part of an observational impact-evaluation design when treatment was not randomly assigned. However, the evaluation should clearly state the assumptions, estimand, matching strategy, balance diagnostics and limitations.

Key Takeaways

PSM is designed for observational data where treatment assignment was not randomized.
The propensity score represents the probability of treatment given observed covariates.
Use appropriate pre-treatment covariates to estimate the propensity score.
Matching is only one stage of the analysis; balance assessment is essential.
Good matching does not eliminate unmeasured confounding.
Document the matching method, estimand, covariates, overlap and observations removed.
Fictional datasets are useful for learning the workflow without exposing real participant information.

PSM Workflow for M&E

Fictional / observational evaluation data
                  ↓
       Define treatment and outcome
                  ↓
       Identify baseline covariates
                  ↓
        Estimate propensity scores
                  ↓
          Select matching method
                  ↓
        Match treatment/comparison
                  ↓
          Assess covariate balance
                  ↓
          Check overlap and losses
                  ↓
       Analyse the matched outcome
                  ↓
       Interpret assumptions and limits

Further Resources