MatchIt Script: Matching and Diagnostics in R for M&E

Matching is one approach for improving comparability between treatment and comparison groups in observational evaluation data. However, creating a matched dataset does not by itself demonstrate that the matching worked.

This tutorial presents a practical MatchIt workflow in R for matching, checking covariate balance, examining propensity-score overlap, reviewing diagnostics, extracting matched data, and documenting the analysis.

Important: Matching can improve comparability on observed characteristics, but it does not recreate randomization or automatically eliminate unmeasured confounding.

What You Will Learn

• Prepare an observational M&E dataset for matching.
• Run nearest-neighbour matching with MatchIt.
• Specify the treatment, covariates and estimand.
• Assess balance before and after matching.
• Create a Love plot.
• Examine propensity-score overlap.
• Examine covariate distributions.
• Identify observations that were not matched.
• Extract the matched dataset.
• Use AI to review MatchIt diagnostics.
• Identify methodological issues that require evaluator judgment.

What Is MatchIt?

MatchIt is an R package for implementing matching and related preprocessing methods for observational data.

It supports several approaches, including nearest-neighbour, optimal, full, exact, coarsened exact and subclassification methods.

For this tutorial, we use 1:1 nearest-neighbour propensity-score matching.

MatchIt documentation

The M&E Scenario

Imagine an NGO implemented a livelihood training programme.

Participation was voluntary rather than randomly assigned.

The evaluation team wants to examine whether participation was associated with higher endline household income.

The dataset contains:

id
treatment
age
education
household_size
baseline_income
distance_km
endline_income

The treatment variable is:

1 = programme participant
0 = comparison participant

The baseline variables are used to estimate treatment assignment and construct a more comparable comparison group.

The outcome is:

endline_income

The outcome should not be used to estimate the propensity score.

Step 1: Install MatchIt

Install the package:

install.packages("MatchIt")

Then load it:

library(MatchIt)

You only need to install the package once.

Step 2: Prepare the Dataset

Suppose your dataset is called:

psm_data

Inspect the variable names:

names(psm_data)

Check the structure:

str(psm_data)

View the first observations:

head(psm_data)

Check treatment coding:

table(psm_data$treatment)

Before proceeding, confirm that:

• The treatment variable is correctly coded.
• Each row represents the appropriate unit of analysis.
• Baseline variables were measured before treatment.
• The outcome is not being used as a matching covariate.

Step 3: Check for Missing Data

Check missing values in the variables used for matching:

colSums(
  is.na(
    psm_data[
      c(
        "treatment",
        "age",
        "education",
        "household_size",
        "baseline_income",
        "distance_km"
      )
    ]
  )
)

Do not automatically remove or impute missing values without considering why the data are missing and how the decision could affect the evaluation.

AI can help you inspect missing-data code, but the final decision about handling missing data requires methodological judgment.

Step 4: Examine the Treatment Groups

Check the number of observations in each group:

table(psm_data$treatment)

Calculate the proportions:

prop.table(
  table(psm_data$treatment)
)

Record these numbers before matching so you can later compare them with the matched sample.

Step 5: Examine Baseline Characteristics

Before matching, examine important baseline variables.

Age:

aggregate(
  age ~ treatment,
  data = psm_data,
  FUN = mean
)

Education:

aggregate(
  education ~ treatment,
  data = psm_data,
  FUN = mean
)

Baseline income:

aggregate(
  baseline_income ~ treatment,
  data = psm_data,
  FUN = mean
)

Household size:

aggregate(
  household_size ~ treatment,
  data = psm_data,
  FUN = mean
)

These comparisons describe the initial differences between the groups.

However, do not select matching variables solely because they show statistically significant differences. Covariate selection should be informed by the evaluation’s causal framework and substantive knowledge.

Step 6: Run Nearest-Neighbour Matching

Set a seed for reproducibility:

set.seed(2026)

Now create the MatchIt model:

match_model <- matchit(
  treatment ~ age +
    education +
    household_size +
    baseline_income +
    distance_km,
  data = psm_data,
  method = "nearest",
  distance = "glm",
  ratio = 1,
  replace = FALSE,
  estimand = "ATT"
)

This specifies:

• treatment as the treatment indicator.
• The other variables as baseline covariates.
• method = "nearest" for nearest-neighbour matching.
• distance = "glm" for propensity-score estimation using a generalized linear model.
• ratio = 1 for one-to-one matching.
• replace = FALSE so comparison observations are not reused.
• estimand = "ATT" to target the Average Treatment Effect on the Treated.

The matching method and estimand should be chosen according to the evaluation design rather than copied automatically from an example.

Step 7: Inspect the Matching Result

Start with:

match_model

Then run:

summary(match_model)

This provides information about the original and matched samples, treatment groups and balance statistics.

Pay particular attention to how many observations remain after matching.

Step 8: Assess Covariate Balance

Create a balance summary:

match_summary <- summary(
  match_model,
  standardize = TRUE,
  improvement = TRUE
)
match_summary

Standardized mean differences make it easier to compare imbalance across variables measured on different scales.

The key question is:

Did matching reduce the differences between the treatment and comparison groups on the observed baseline covariates?

Do not interpret a single threshold as an automatic pass/fail rule. Balance should be assessed in the context of the variables, evaluation design and intended analysis.

Step 9: Create a Love Plot

A Love plot provides a visual summary of standardized mean differences before and after matching.

plot(
  match_summary,
  var.order = "unmatched",
  threshold = 0.1
)

The plot helps you identify variables for which matching substantially reduced imbalance.

A standardized mean difference closer to zero generally indicates better balance.

A threshold such as 0.1 is commonly used as a diagnostic reference, but it should not be treated as a universal rule that automatically establishes adequate balance.

Step 10: Examine Propensity-Score Overlap

Matching also requires sufficient overlap between treated and comparison observations.

Examine the propensity-score distributions:

plot(
  match_model,
  type = "histogram"
)

You can also use a jitter plot:

plot(
  match_model,
  type = "jitter",
  interactive = FALSE
)

The purpose is to determine whether treated and comparison observations have sufficient common support for matching.

Step 11: Examine Covariate Distributions

Standardized mean differences focus on differences in means. It can also be useful to examine the distributions of individual covariates.

plot(
  match_model,
  type = "density",
  which.xs = ~ age +
    education +
    baseline_income,
  interactive = FALSE
)

You can also examine empirical cumulative distributions:

plot(
  match_model,
  type = "ecdf",
  which.xs = ~ age +
    education +
    baseline_income,
  interactive = FALSE
)

These diagnostics can reveal distributional differences that may not be obvious from mean comparisons alone.

Step 12: Extract the Matched Dataset

Once you have reviewed the diagnostics, extract the matched observations:

matched_data <- match.data(
  match_model
)

Inspect the matched data:

head(matched_data)

Check the treatment groups:

table(
  matched_data$treatment
)

Check the matched sample size:

nrow(matched_data)

Inspect the matching weights:

summary(
  matched_data$weights
)

Step 13: Check How Many Observations Were Lost

Compare the original sample size with the matched sample:

nrow(psm_data)
nrow(matched_data)

You should also inspect:

summary(match_model)

If observations were discarded or left unmatched, document this clearly.

A large number of discarded observations can change the population to which the analysis applies.

Step 14: Recheck the Matched Groups

For example, compare age after matching:

aggregate(
  age ~ treatment,
  data = matched_data,
  FUN = mean
)

Education:

aggregate(
  education ~ treatment,
  data = matched_data,
  FUN = mean
)

Baseline income:

aggregate(
  baseline_income ~ treatment,
  data = matched_data,
  FUN = mean
)

These simple comparisons can help explain the matched sample, but the formal balance diagnostics should remain the primary basis for assessing balance.

Step 15: Conduct a Sensitivity Analysis With a Caliper

You can examine whether a caliper changes the matching results.

match_model_caliper <- matchit(
  treatment ~ age +
    education +
    household_size +
    baseline_income +
    distance_km,
  data = psm_data,
  method = "nearest",
  distance = "glm",
  ratio = 1,
  replace = FALSE,
  estimand = "ATT",
  caliper = 0.2,
  std.caliper = TRUE
)

Assess balance again:

caliper_summary <- summary(
  match_model_caliper,
  standardize = TRUE,
  improvement = TRUE
)
caliper_summary

Create the Love plot:

plot(
  caliper_summary,
  var.order = "unmatched",
  threshold = 0.1
)

A caliper should be justified as part of the analytical strategy rather than introduced simply because it produces a preferred result.

Step 16: Compare Matching Specifications

Compare the original and caliper specifications:

summary(match_model)
summary(match_model_caliper)

Consider:

• Did balance improve?
• How many observations were retained?
• Did overlap improve?
• Did the target population change?
• Are the conclusions sensitive to the matching specification?

Do not repeatedly modify the model until you obtain a preferred result.

Step 17: Use AI to Review MatchIt Diagnostics

AI can help explain technical output and identify issues for further investigation.

For example:

You are reviewing a propensity score matching analysis
for an M&E evaluation.
I will provide:
1. MatchIt summary output
2. Standardized mean differences
3. Love plot
4. Propensity-score overlap plot
5. Matched sample size
Review the diagnostics.
Report:
A. What improved after matching
B. What remains imbalanced
C. Signs of limited overlap
D. How many observations were lost
E. Important limitations
F. Questions the evaluator should investigate
Do not state that matching proves causality.
Do not invent information.
Do not recommend changing the model solely to obtain
a more favourable treatment effect.
Clearly distinguish statistical observations
from methodological judgment.

This creates a useful division of responsibilities:

R / MatchIt
    ↓
Calculations and diagnostics
    ↓
AI
    ↓
Explanation and review
    ↓
Evaluator
    ↓
Methodological decisions

Step 18: Ask AI to Audit the R Script

AI can also review the actual R code.

Review this MatchIt R script as a statistical programming
and M&E methodology reviewer.
Check for:
1. R syntax errors
2. Incorrect MatchIt arguments
3. Incorrect variable names
4. Post-treatment variables
5. Missing-data problems
6. Incorrect treatment coding
7. Missing balance diagnostics
8. Missing overlap diagnostics
9. Problems with interpretation
10. Potential causal-inference issues
For every issue, explain:
- what is wrong
- why it matters
- how it can be corrected
Do not claim that matching establishes causality.

Step 19: Use AI to Document the Analysis

After completing the analysis, AI can help turn the code and diagnostic output into a reproducibility note.

Using only the information in the MatchIt script
and diagnostic output below, create a reproducibility
note for an evaluation report.
Include:
- treatment definition
- comparison definition
- covariates
- matching method
- propensity-score model
- estimand
- replacement setting
- matching ratio
- caliper, if used
- original sample size
- matched sample size
- balance findings
- overlap findings
- observations excluded
- limitations
Do not invent missing information.
Flag anything that needs to be added manually.

What Good Diagnostics Look Like

A defensible matching workflow should provide evidence that:

• Important baseline differences were reduced.
• Standardized mean differences became smaller.
• The matched sample has reasonable overlap.
• Important covariates are adequately balanced.
• The number of discarded observations is understood.
• The matched population is clearly identified.
• The analysis is consistent with the intended estimand.

The important question is not:

“Did the MatchIt script run successfully?”

The important question is:

“Did the matching procedure produce a defensible comparison group for the evaluation question?”

What Diagnostics Cannot Tell You

Even strong balance diagnostics cannot establish that:

• All important confounders were measured.
• Unobserved confounding is absent.
• The treatment was correctly defined.
• The outcome was measured without bias.
• There was no differential attrition.
• The causal model is correct.

Matching diagnostics primarily address comparability on observed variables.

They do not prove causal validity.

Common MatchIt Mistakes

Running Matching Without Checking Balance

Do not stop after running:

match_model <- matchit(...)

Always examine balance.

Treating Equal Sample Sizes as Evidence of Balance

A 1:1 matched sample can still have substantial covariate imbalance.

Matching on Post-Treatment Variables

Avoid variables that could have been affected by the intervention.

Selecting Covariates Based Only on P-Values

Matching variables should not be selected solely because their baseline differences are statistically significant.

Ignoring Overlap

Limited common support can make matching difficult and can affect the population to which the analysis applies.

Ignoring Discarded Observations

Always document observations that were not matched or were discarded.

Changing the Model Until the Result Looks Good

Repeatedly changing matching specifications to obtain a preferred result can introduce analytical bias.

Assuming a Love Plot Proves Causal Validity

A Love plot displays balance. It does not demonstrate that all confounding has been eliminated.

Ignoring the Estimand

Make clear whether the analysis targets the ATT, ATE or another estimand.

Treating MatchIt as a Replacement for Evaluation Design

MatchIt implements a statistical procedure. It does not determine the evaluation design or causal assumptions.

Practical M&E Interpretation

Suppose an evaluation begins with:

Treatment:      350
Comparison:     650

After matching:

Treatment:      350
Comparison:     350

Suppose standardized mean differences change from:

Age              0.31 → 0.05
Education        0.28 → 0.04
Baseline income  0.44 → 0.07
Distance         0.22 → 0.06

This suggests that matching substantially improved balance on these observed covariates.

An appropriate interpretation would be:

“The matching procedure substantially improved observed baseline comparability on the included covariates.”

It would be inappropriate to conclude:

“Matching proved that the programme caused the observed difference.”

The second statement goes beyond what the diagnostics establish.

Reporting MatchIt Results

A concise methodological description could look like this:

The evaluation used 1:1 nearest-neighbour propensity-score matching to improve comparability between programme participants and eligible non-participants. Propensity scores were estimated using baseline age, education, household size, baseline income and distance from the training centre. Balance was assessed using standardized mean differences before and after matching, together with graphical diagnostics of covariate distributions and propensity-score overlap. The matched sample was then used for the outcome analysis. The results should be interpreted in light of the assumption that relevant confounding variables were adequately measured and included in the matching model.

Adapt this description to the actual analysis rather than copying it automatically.

Complete MatchIt Workflow

Evaluation question
        ↓
Define treatment and comparison
        ↓
Identify pre-treatment covariates
        ↓
Check data quality
        ↓
Run MatchIt
        ↓
Check matching sample
        ↓
Assess standardized mean differences
        ↓
Create Love plot
        ↓
Check propensity-score overlap
        ↓
Check covariate distributions
        ↓
Review observations lost
        ↓
Assess whether balance improved
        ↓
Extract matched dataset
        ↓
Conduct outcome analysis
        ↓
Interpret assumptions and limitations
        ↓
Document the analysis

Key Takeaways

1. MatchIt can implement several matching strategies for observational evaluation data.
2. Nearest-neighbour matching is only one possible approach.
3. Always assess balance before and after matching.
4. Standardized mean differences are useful for assessing covariate balance.
5. Love plots provide a visual summary of balance.
6. Propensity-score overlap should also be examined.
7. Distributional diagnostics can reveal problems that mean comparisons may miss.
8. Always document observations that were unmatched or discarded.
9. Sensitivity analyses can help assess whether results depend on the matching specification.
10. AI can help explain, audit and document MatchIt analyses.
11. AI should not make unsupported causal or methodological decisions.
12. Improved balance on observed variables does not prove that the comparison is unbiased.
13. The final methodological interpretation remains the responsibility of the evaluator.

Further Resources

MatchIt Documentation

MatchIt: Assessing Balance

MatchIt: Balance Summary

MatchIt: Balance and Overlap Plots

EvalCommunity Academy: AI in Monitoring & Evaluation