Unit 2: Exploring Two-Variable Data

Linear Regression

Fitting a least-squares regression line and assessing it with residuals and r².

Advanced25 min lesson3 min readUpdated August 12, 2026Author not yet attributed

Prerequisites

Two-Way Tables

The Line That Minimizes the Misses

Hours studied and exam score for 6 students: (1,65), (2,70), (3,80), (4,82), (5,88), (6,95). Before reading on: infinitely many lines could be drawn through this scatter of points — what would make one particular line the 'best' summary of the relationship?

Definition — Least-Squares Regression Line

The least-squares regression line ŷ = b₀ + b₁x is the one line that minimizes the sum of squared residuals, Σ(y − ŷ)², among every possible line. Squaring each residual before summing (rather than just adding raw residuals) keeps positive and negative errors from cancelling out, and penalizes large misses more heavily than small ones — pulling the line toward avoiding big errors specifically.

Worked Example — Computing the Regression Line by Hand

With x̄ = 3.5 and ȳ = 80, computing Σ(x−x̄)(y−ȳ) = 103 and Σ(x−x̄)² = 17.5 gives slope b₁ = 103/17.5 ≈ 5.886, and intercept b₀ = ȳ − b₁x̄ = 80 − 5.886(3.5) = 59.4. The regression line is ŷ = 59.4 + 5.886x. The slope means: for each additional hour studied, predicted exam score increases by about 5.9 points on average. The intercept, 59.4, would be the predicted score for 0 hours studied — but since the data only covers 1 to 6 hours, using the intercept this way is extrapolation, predicting outside the range the data actually supports.

b_1 = \frac{\sum (x-\bar{x})(y-\bar{y})}{\sum (x-\bar{x})^2}

Worked Example — Assessing the Fit with Residuals and r²

Predicted values: ŷ(1)≈65.29, ŷ(2)≈71.17, ŷ(3)≈77.06, ŷ(4)≈82.94, ŷ(5)≈88.83, ŷ(6)≈94.71. Residuals (actual − predicted): −0.29, −1.17, 2.94, −0.94, −0.83, 0.29 — summing to 0, as a correctly computed least-squares line's residuals always do. None of the residuals shows an obvious curved pattern, supporting a linear model as reasonable, though 6 points is a small sample to judge from confidently. Computing r² from Σ(y−ȳ)² = 618: r² = (103)²/(17.5×618) ≈ 0.981 — about 98.1% of the variation in exam score is explained by its linear relationship with hours studied, and r = √0.981 ≈ 0.990, a very strong positive linear association.

Graph Visualizer

Trace modePoint mode (click graph to plot)

Show x^2

Color

Expression

Domain & range

x min

x max

y min

y max

Precision (decimal places)2

Evaluate a point

Evaluate at x =

x^2 = 0

Tip

r² measures how well the line fits — it says nothing about whether studying more actually causes higher scores. This is observational data on hours studied and scores, not a randomized experiment, so only association is established here.

Common Mistakes

Using the regression line to predict a value far outside the range of x actually observed.
A regression line is only trustworthy for interpolation (predicting within the observed range of x) — extrapolating beyond it, like predicting the score for 0 or 20 hours studied here, assumes the linear pattern continues, which the data can't confirm.
Reporting r² as a percentage of something other than variation explained, e.g. 'the model is correct 98% of the time.'
r² specifically measures the proportion of variation in y explained by the linear relationship with x — it isn't an accuracy rate or a probability of a correct prediction.

Key Takeaways

The least-squares regression line minimizes the sum of squared residuals, which is why residuals always sum to exactly 0 for a correctly computed line.
The slope and intercept have specific context-based interpretations; using the intercept for x-values outside the observed data range is extrapolation.
r² measures the proportion of variation in y explained by the linear relationship — it does not establish causation.

Summary

This closes Unit 2. Unit 3 turns from describing already-collected data to how that data should be collected in the first place, so its conclusions can actually be trusted.

Track your progress

← Two-Way Tables Back to Unit 2: Exploring Two-Variable Data