2.4Fitting Linear Models to Data
Draw scatter plots, identify linear trends, find lines of best fit using least-squares regression, interpret the correlation coefficient r, and distinguish correlation from causation.
Linear regression is one of the most widely used statistical tools in science, business, and medicine. Understanding how to fit a line to data and interpret the correlation coefficient is a foundational data literacy skill.
Essential Question
How do you find the line that best fits a set of data points, and how do you measure how well that line actually fits?
Lesson Overview
A scatter plot displays paired data (x, y) as individual points on a coordinate plane. When the points suggest a linear trend, we fit a line of best fit (also called a regression line or least-squares line) to the data. The least-squares regression line ŷ = mx + b minimizes the sum of the squared residuals — the vertical distances between each data point and the line. The formulas are: m = [n·Σ(xy) − Σx·Σy] / [n·Σ(x²) − (Σx)²] and b = (Σy − m·Σx) / n. The correlation coefficient r (−1 ≤ r ≤ 1) measures the strength and direction of the linear relationship. |r| close to 1 means strong linear correlation; |r| close to 0 means weak or no linear correlation. The sign of r matches the sign of the slope. A residual is the difference between the actual y-value and the predicted ŷ-value: residual = y − ŷ. Positive residuals lie above the line; negative residuals lie below. The coefficient of determination r² tells what fraction of the variation in y is explained by the linear model. Finally, correlation does not imply causation: two variables can be strongly correlated without one causing the other — a lurking variable may explain both.
Least-Squares Formulas
Given n data points (x₁,y₁), …, (xₙ,yₙ):
m = [n·Σ(xy) − Σx·Σy] / [n·Σ(x²) − (Σx)²]
b = (Σy − m·Σx) / n
Or equivalently: b = ȳ − m·x̄
where x̄ = Σx/n and ȳ = Σy/n (the means)
Key Facts about r
- Always: −1 ≤ r ≤ 1
- r = 1: perfect positive linear correlation
- r = −1: perfect negative linear correlation
- r = 0: no linear correlation
- Sign of r = sign of slope m
- r² = fraction of variation explained
- |r| ≥ 0.8: strong; 0.5–0.8: moderate; <0.5: weak
Types of Correlation
Strong Positive
r ≈ 0.95
Strong Negative
r ≈ −0.93
Weak Positive
r ≈ 0.55
No Correlation
r ≈ 0.02
Correlation Coefficient r — Interpretation Scale
|r| ≥ 0.8 — Strong
Points cluster tightly around the line. Model is reliable.
0.5 ≤ |r| < 0.8 — Moderate
Noticeable trend but significant scatter. Use with caution.
|r| < 0.5 — Weak/None
Little or no linear relationship. Linear model is not appropriate.
Residuals: Actual − Predicted
Residual = Actual y − Predicted ŷ. Least-squares minimizes the sum of squared residuals.
Worked Examples
The data shows hours of TV watched per day (x) and GPA (y): (1, 3.8), (2, 3.5), (3, 3.1), (4, 2.8), (5, 2.4). Describe the scatter plot and the correlation.
Plot the five points on a coordinate plane with x-axis = TV hours, y-axis = GPA.
As x increases, y decreases — the points slope downward left to right.
The points lie close to a straight line with no obvious curve.
Correlation: strong negative linear correlation. r is close to −1.
Note: this is correlation, not causation — other factors may explain both variables.
Use the least-squares formulas to find the regression line for: (1, 3), (2, 5), (3, 4), (4, 7), (5, 8). Show all work.
n = 5. Build a table: Σx = 1+2+3+4+5 = 15, Σy = 3+5+4+7+8 = 27.
Σ(x²) = 1+4+9+16+25 = 55. Σ(xy) = 1·3+2·5+3·4+4·7+5·8 = 3+10+12+28+40 = 93.
m = [5·93 − 15·27] / [5·55 − 15²] = [465 − 405] / [275 − 225] = 60/50 = 1.2.
b = (27 − 1.2·15)/5 = (27 − 18)/5 = 9/5 = 1.8.
Regression line: ŷ = 1.2x + 1.8.
Check with means: x̄ = 3, ȳ = 5.4. b = 5.4 − 1.2(3) = 5.4 − 3.6 = 1.8 ✓.
For the regression line ŷ = 1.2x + 1.8 from Example 2, find the residuals for each data point. Which point is farthest from the line?
Residual = actual y − predicted ŷ.
x=1: ŷ=1.2(1)+1.8=3.0, residual=3−3.0=0.
x=2: ŷ=1.2(2)+1.8=4.2, residual=5−4.2=+0.8.
x=3: ŷ=1.2(3)+1.8=5.4, residual=4−5.4=−1.4.
x=4: ŷ=1.2(4)+1.8=6.6, residual=7−6.6=+0.4.
x=5: ŷ=1.2(5)+1.8=7.8, residual=8−7.8=+0.2.
Largest |residual| = 1.4 at x = 3. That point is farthest from the line.
A study finds r = 0.92 between ice cream sales and drowning rates. Does eating ice cream cause drowning? Explain using the concept of a lurking variable.
r = 0.92 indicates a strong positive linear correlation between the two variables.
However, correlation does NOT imply causation.
A lurking variable — hot weather — explains both: hot weather increases both ice cream sales AND swimming activity (which increases drowning risk).
The two variables are correlated because they share a common cause, not because one causes the other.
Conclusion: we cannot conclude that ice cream causes drowning, despite the high r value.
A regression line for advertising spend x (thousands $) vs. sales y (thousands $) is ŷ = 4.5x + 12, with r² = 0.81. Interpret the slope, y-intercept, and r².
Slope 4.5: for each additional $1,000 spent on advertising, sales increase by $4,500.
y-intercept 12: when advertising spend is $0, predicted sales are $12,000 (baseline sales).
r² = 0.81: 81% of the variation in sales is explained by the linear relationship with advertising spend.
The remaining 19% of variation is due to other factors not in the model.
r = √0.81 = 0.9 (positive, since slope is positive): strong positive correlation.
Guided Practice
Describe the correlation for each scatter plot description: (a) points cluster tightly along a rising line, (b) points are scattered randomly with no pattern, (c) points loosely follow a falling line.
Hint: For each: identify direction (positive/negative/none) and strength (strong/weak). Match to r values: (a) r close to +1, (b) r close to 0, (c) r moderate negative.
Find the regression line for: (1, 2), (2, 4), (3, 5), (4, 4), (5, 7). Use the least-squares formulas.
Hint: Build a table: find n, Σx, Σy, Σ(x²), Σ(xy). Then m = [nΣ(xy) − ΣxΣy] / [nΣ(x²) − (Σx)²] and b = (Σy − mΣx)/n.
A regression line is ŷ = 3x − 1. The actual data point is (4, 14). Find the residual and state whether the point is above or below the line.
Hint: Residual = actual y − predicted ŷ. Compute ŷ at x = 4. Positive residual → above line; negative → below.
A study finds r = −0.73 between hours of sleep and number of errors on a test. Interpret r. Is this strong, moderate, or weak? What does the sign tell you?
Hint: |r| = 0.73 falls in the moderate range (0.5–0.8). Negative sign means as sleep increases, errors decrease. State this in context.
A regression model for temperature (x, °F) vs. energy bills (y, $) has r² = 0.64. What percentage of variation in energy bills is explained by temperature? What does the remaining percentage represent?
Hint: r² = 0.64 means 64% explained. The remaining 36% is due to other factors (insulation quality, appliance use, etc.).
Key Vocabulary
Scatter Plot
A graph of paired (x, y) data points. Used to visually identify trends, clusters, and outliers in bivariate data.
Line of Best Fit (Regression Line)
The line ŷ = mx + b that minimizes the sum of squared residuals. Also called the least-squares line.
Correlation Coefficient (r)
A number from −1 to 1 measuring the strength and direction of a linear relationship. r = ±1 is perfect; r = 0 is no linear correlation.
Residual
The difference between an actual y-value and the predicted ŷ-value: residual = y − ŷ. Positive = above line; negative = below line.
Coefficient of Determination (r²)
The square of r. Represents the fraction of variation in y explained by the linear model. r² = 0.81 means 81% explained.
Lurking Variable
A variable not included in the analysis that influences both x and y, creating a spurious correlation. Explains why correlation ≠ causation.
Interactive Practice — 5 Questions
A scatter plot shows points rising steeply from left to right, clustered tightly around a line. Which r value best describes this?
For data (1,4), (2,6), (3,8), (4,10), (5,12), what is the regression line?
The regression line is ŷ = 5x + 3. The actual point is (2, 15). What is the residual?
A model has r² = 0.76. What percentage of variation in y is NOT explained by the linear model?
Cities with more fire stations tend to have more fires. This means fire stations cause fires.
Independent Practice
Independent Practice
Describe the type of correlation (strong positive, moderate negative, none, etc.) you would expect between: (a) height and shoe size, (b) hours of exercise and body fat %, (c) hair color and IQ score.
Find the least-squares regression line for: (2, 5), (4, 9), (6, 11), (8, 15), (10, 19). Show all steps.
The regression line ŷ = −2x + 30 has r = −0.88. Interpret the slope, y-intercept, and r in context of: x = temperature (°C) and y = hot chocolate sales (cups/day).
A data set has r² = 0.49. (a) Find r (assume positive correlation). (b) Is the linear model a good fit? (c) What does the remaining variation represent?
Identify a possible lurking variable for each: (a) shoe size and reading ability in children, (b) number of hospitals and death rate in cities, (c) ice cream sales and sunburn cases.
Common Mistakes
Concluding causation from a high r value: 'r = 0.95, so x causes y.'
Correlation measures the strength of a linear relationship, not causation. Always ask: could a lurking variable explain both? Could the relationship be coincidental?
Confusing r and r²: saying 'r² = 0.81 means r = 0.81.'
r² = 0.81 means r = ±0.9 (take the square root). The sign of r matches the sign of the slope. r and r² are different measures with different interpretations.
Using the regression line to predict far outside the data range without noting the risk.
Extrapolation with a regression line is unreliable. The linear trend may not hold outside the data range. Always note when a prediction is an extrapolation.
Forgetting that the regression line always passes through (x̄, ȳ) — the point of means.
The least-squares line always passes through the means of x and y. Use this as a quick check: substitute x̄ into ŷ = mx + b and verify you get ȳ.
Math Tips
Shortcut for the regression line: compute x̄ and ȳ first. Then m = Σ[(xᵢ − x̄)(yᵢ − ȳ)] / Σ[(xᵢ − x̄)²]. Finally b = ȳ − mx̄. The line always passes through (x̄, ȳ).
Memory trick for r: "r is for relationship." Its sign tells direction (+ or −); its magnitude tells strength (closer to ±1 = stronger). r² tells how much of the story the line explains.
When the data is perfectly linear (all points on one line), r = ±1 and all residuals are 0. Real data is never perfect, so r = ±1 is a theoretical ideal.
A residual plot (residuals vs. x) should look random with no pattern if the linear model is appropriate. A curved pattern in the residuals suggests a non-linear model would fit better.
On the SAT/ACT, "correlation coefficient" questions often ask you to match a scatter plot to an r value. Key: direction (sign) + tightness of clustering (magnitude). Practice all four quadrants of the correlation grid.