Difference-in-Differences Explained: Measuring Policy Effects with Natural Experiments
- ▸Difference-in-differences compares the change in a treated group with the change in a similar untreated group, cancelling shared shocks.
- ▸In the 1992 New Jersey minimum wage study, employment rose slightly in New Jersey and fell in Pennsylvania, a DiD estimate of about +2.75 jobs per restaurant.
- ▸The estimate is causal only under parallel trends; standard errors should be clustered by group, and staggered adoption needs newer estimators.
Understanding difference-in-differences — simply put
Difference-in-differences (DiD) estimates the effect of a policy by comparing how an outcome changed in a group exposed to the policy with how it changed, over the same period, in a similar group that was not. Subtracting the second change from the first removes everything the two groups have in common, such as the business cycle or seasonal patterns, and leaves the effect of the policy itself.
Why not just compare before and after?
A before-and-after comparison attributes everything that changed over the period to the policy, including recessions, seasons and national trends. A comparison between treated and untreated groups at one point in time attributes every pre-existing difference between them to the policy. DiD uses each comparison to cancel the bias in the other.
A real example: did a higher minimum wage cost jobs?
In April 1992 New Jersey raised its minimum wage from $4.25 to $5.05 an hour. Neighboring eastern Pennsylvania kept the federal $4.25. Economists David Card and Alan Krueger surveyed fast-food restaurants in both states in February 1992, before the increase, and again in November and December, after it. Standard theory predicted New Jersey employment would fall.
| Group | Before (Feb 1992) | After (Nov–Dec 1992) | Change |
|---|---|---|---|
| New Jersey (treated) | 20.44 | 21.03 | +0.59 |
| Pennsylvania (control) | 23.33 | 21.17 | -2.16 |
| Difference-in-differences | +2.75 |
Employment fell in Pennsylvania, where nothing changed in the law, and rose slightly in New Jersey. Subtracting the control group's change from the treated group's gives an effect of about +2.75 full-time-equivalent jobs per restaurant: no evidence that the higher minimum wage reduced employment.
- New Jersey (minimum wage raised)21.03 FTE employees per restaurant
- Pennsylvania (no change, control)21.17 FTE employees per restaurant
- New Jersey if it had followed Pennsylvania's trend18.28 FTE employees per restaurant
View data
| Series | Date | Value |
|---|---|---|
| New Jersey (minimum wage raised) | 1992-02-15 | 20.44 FTE employees per restaurant |
| New Jersey (minimum wage raised) | 1992-11-15 | 21.03 FTE employees per restaurant |
| Pennsylvania (no change, control) | 1992-02-15 | 23.33 FTE employees per restaurant |
| Pennsylvania (no change, control) | 1992-11-15 | 21.17 FTE employees per restaurant |
| New Jersey if it had followed Pennsylvania's trend | 1992-02-15 | 20.44 FTE employees per restaurant |
| New Jersey if it had followed Pennsylvania's trend | 1992-11-15 | 18.28 FTE employees per restaurant |
What has to be true for the estimate to be causal?
The key assumption is parallel trends: without the policy, the treated group's outcome would have moved in step with the control group's. It cannot be tested directly, because the counterfactual is never observed, but it can be made credible by showing that the two groups moved together over several periods before the policy, and by choosing a control group exposed to the same shocks. Pennsylvania restaurants shared New Jersey's regional economy, which is why the design was persuasive.
The same estimate comes from a regression with group and period effects:
The regression form allows control variables, many groups and many periods, and gives standard errors directly; with data on several periods those should be clustered by group, because outcomes within a group are correlated over time.
Why the debate did not end there
Later studies using payroll records instead of telephone surveys found smaller or even negative effects, and the minimum-wage literature remains contested. DiD is only as good as its parallel-trends assumption and its data. Recent work has also shown that the standard two-way regression can be biased when different groups adopt a policy at different times, which has led to new estimators for staggered designs.
The takeaway
Difference-in-differences turns a natural experiment into a causal estimate by comparing changes, not levels. Its credibility rests on parallel trends: check pre-policy trends, choose a control group that faces the same shocks, and cluster the standard errors.
Further reading
- OLS Regression Explained — internal
- Time Series Analysis Explained — internal
- Card and Krueger (1994), Minimum Wages and Employment, American Economic Review — external reference
Frequently Asked Questions
What is difference-in-differences?
Difference-in-differences is a method for estimating a policy effect by comparing the change in an outcome for a group affected by the policy with the change over the same period for a similar group that was not affected. The difference between the two changes is the estimated effect.
What is the parallel trends assumption?
Parallel trends means that, without the policy, the treated and control groups would have followed the same path over time. It cannot be tested directly but is supported when the two groups moved together in the periods before the policy.
How do you run difference-in-differences as a regression?
Regress the outcome on a treated-group indicator, a post-period indicator and their interaction. The coefficient on the interaction term is the difference-in-differences estimate, and standard errors should be clustered at the group level.
What did Card and Krueger find about the minimum wage?
Comparing fast-food restaurants in New Jersey, which raised its minimum wage in 1992, with those in neighboring Pennsylvania, Card and Krueger found no reduction in employment; their difference-in-differences estimate was a small increase. Later studies using payroll data reached more mixed conclusions.
Primary Sources
Cite This Article
EconoLens Research Desk. (2026, October 11). Difference-in-Differences Explained: Measuring Policy Effects with Natural Experiments. EconoLens. https://www.econolens.co.in/news/study-difference-in-differences-explained
The EconoLens Research Desk reviews academic papers in economics and econometrics, translating cutting-edge research into accessible analysis. Full credit is given to original authors in every review.