Understanding OLS Regression: How Economists Estimate Real Relationships

Understanding OLS regression — simply put

Ordinary least squares (OLS) estimates how much one variable changes, on average, when another changes by one unit. It fits the straight line through a cloud of data points that makes the squared vertical distances from the points to the line as small as possible. Almost every empirical claim in economics, from the return to a year of schooling to the effect of a rate rise on inflation, starts with an OLS regression.

Why square the errors rather than add them?

Positive and negative errors would cancel if simply added, so the line must penalize both. Squaring does that, gives a unique solution with a closed form, and penalizes large misses more than small ones. The payoff is the Gauss–Markov theorem: when the errors have mean zero, constant variance and no correlation with each other or with X, OLS is the best linear unbiased estimator. No other linear, unbiased method gives estimates with smaller variance.

How do you read a regression table?

The table regresses weekly earnings on years of schooling for ten workers. The sample is a teaching example built to resemble U.K. survey patterns; every statistic is computed exactly from it.

The slope says each extra year of schooling is associated with £44 more a week. Its standard error of 2.39 measures sampling uncertainty; the t-statistic of 18.4 is far above 2, so the slope is clearly different from zero. The intercept of −£43.80 is an extrapolation to zero years of schooling, outside the data, and has no meaning on its own. An R² of 0.977 says schooling accounts for almost all the variation in this small sample; real survey data, with thousands of noisy observations, typically give R² values far lower without the slope being any less useful.

When does OLS give the wrong answer with confidence?

The most dangerous failure is omitted variable bias. If a variable that affects Y is left out and is correlated with X, its effect is loaded onto the slope:

Ability is the classic example. More able people tend to stay in school longer and also earn more, so a regression of earnings on schooling alone overstates the return to schooling. A larger sample does not fix this: the estimate converges to the wrong number. Fixing it requires a different design, such as instrumental variables, panel data or a natural experiment.

What does a regression on economy-wide data look like?

Okun's law is one of the most famous OLS relationships in macroeconomics: when real GDP grows faster than its trend, unemployment falls. Regressing the change in the unemployment rate on real GDP growth for the United States has historically produced a slope of roughly −0.5: each extra point of growth goes with about half a point less unemployment.

Time-series regressions like this need extra care. Errors are usually correlated over time, which makes ordinary standard errors too small; economists use heteroskedasticity- and autocorrelation-robust (Newey–West) standard errors instead.

The takeaway

OLS gives the best linear unbiased estimate when its assumptions hold, and a precise, confident, wrong one when an omitted variable is correlated with the regressor. Read the slope, its standard error and its economic size, and always ask what the regression has left out.

Further reading

• Time Series Analysis Explained — internal
• Difference-in-Differences Explained — internal
• FRED: Unemployment rate (UNRATE) — external reference