In Lecture 5, we used bins to summarize how players’ batting averages changed from 2014 to 2015. The bin averages described the pattern in the data. Linear regression gives us a single smooth line for that pattern, without requiring us to choose bin widths.

Load the data (you can download it here) and reshape it so that each player has separate columns for 2014 and 2015.

load("data/batting_2014_2015.RData")

batting_2014_2015 <- batting_2014_2015 %>%
  pivot_wider(
    names_from = yearID,
    values_from = BA,
    names_prefix = "BA_"
  )

head(batting_2014_2015)
## # A tibble: 6 × 3
##   playerID  BA_2014 BA_2015
##   <chr>       <dbl>   <dbl>
## 1 abreujo02   0.317   0.290
## 2 altuvjo01   0.341   0.313
## 3 andruel01   0.263   0.258
## 4 aybarer01   0.278   0.270
## 5 bautijo02   0.286   0.250
## 6 beltrad01   0.324   0.287
ggplot(batting_2014_2015, aes(x = BA_2014, y = BA_2015)) +
  geom_point() +
  labs(x = "2014 Batting Average", y = "2015 Batting Average")

Correlation

The correlation coefficient, r, summarizes the direction and strength of a linear relationship. It ranges from -1 to 1.

  • A correlation near 1 describes a strong positive linear relationship.
  • A correlation near -1 describes a strong negative linear relationship.
  • A correlation near 0 describes a weak linear relationship.

We calculate a correlation with cor(). The function expects two vectors, so we use $ to extract each column.

cor(batting_2014_2015$BA_2014, batting_2014_2015$BA_2015)
## [1] 0.4928799

There is a positive relationship between the two seasons, but there is also considerable variation among players with similar 2014 batting averages.

Correlation measures only linear dependence. In the data below, y is determined exactly by x, but the relationship is curved and the correlation is near zero.

Fitting a Linear Model

Use lm() to fit the line relating 2014 batting average to 2015 batting average.

batting_model <- lm(BA_2015 ~ BA_2014, data = batting_2014_2015)
batting_model$coefficients
## (Intercept)     BA_2014
##   0.1409779   0.4851417

The formula has the form response ~ predictor. Here, BA_2015 is the response on the vertical axis and BA_2014 is the predictor on the horizontal axis.

A fitted line has an intercept and a slope:

y^=a+bx\widehat{y} = a + bx

The intercept, a, is the fitted value when the predictor is zero. That is far outside the batting averages in these data, so it is not substantively useful here.

The slope, b, is the fitted change in 2015 batting average associated with a one-unit increase in 2014 batting average. Since a one-unit change is enormous for batting average, it is clearer to interpret a 0.010 increase.

coef(batting_model)["BA_2014"] * 0.010
##     BA_2014
## 0.004851417

Question: According to the model, how much higher is the fitted 2015 batting average when the 2014 batting average is 10 points higher?

Fitted Values

predict() returns the fitted value for every row used to estimate the model. Add those values to the data and draw the line.

batting_2014_2015 <- batting_2014_2015 %>%
  mutate(fitted = predict(batting_model))

ggplot(arrange(batting_2014_2015, BA_2014)) +
  geom_point(aes(x = BA_2014, y = BA_2015)) +
  geom_line(
    aes(x = BA_2014, y = fitted),
    color = "lightcoral",
    linewidth = 1
  ) +
  labs(
    x = "2014 Batting Average",
    y = "2015 Batting Average",
    title = "Batting Average with Fitted Regression Line"
  )

These are fitted values for players whose 2015 batting averages are already in the data. They summarize the average relationship between the two seasons; they are not out-of-sample forecasts.

For example, the fitted value for a 2014 batting average of 0.310 is:

predict(batting_model, newdata = tibble(BA_2014 = 0.310))
##         1
## 0.2913719

Residuals and Model Fit

A residual is the observed value minus the fitted value. A positive residual means the observed 2015 batting average is above the line; a negative residual means it is below the line.

batting_2014_2015 <- batting_2014_2015 %>%
  mutate(residual = BA_2015 - fitted)

head(batting_2014_2015)
## # A tibble: 6 × 5
##   playerID  BA_2014 BA_2015 fitted residual
##   <chr>       <dbl>   <dbl>  <dbl>    <dbl>
## 1 abreujo02   0.317   0.290  0.295 -0.00417
## 2 altuvjo01   0.341   0.313  0.306  0.00711
## 3 andruel01   0.263   0.258  0.269 -0.0103
## 4 aybarer01   0.278   0.270  0.276 -0.00638
## 5 bautijo02   0.286   0.250  0.280 -0.0291
## 6 beltrad01   0.324   0.287  0.298 -0.0108
ggplot(batting_2014_2015, aes(x = fitted, y = residual)) +
  geom_hline(yintercept = 0, color = "lightcoral", linewidth = 1) +
  geom_point() +
  labs(x = "Fitted 2015 Batting Average", y = "Residual")

A useful residual plot has points scattered around zero without a clear pattern. A curve or funnel shape would tell us that a straight line or constant spread does not describe the data well.

Two numbers summarize different aspects of model fit:

  • R2R^2 is the proportion of variation in the response explained by the fitted line.
  • RMSE is the typical size of a residual, in the same units as the response.
summary(batting_model)$r.squared
## [1] 0.2429305
batting_2014_2015 %>%
  summarize(RMSE = sqrt(mean(residual^2)))
## # A tibble: 1 × 1
##     RMSE
##    <dbl>
## 1 0.0223

For a linear regression with one predictor, R2R^2 equals the squared correlation. A larger R2R^2 means the line explains more of the variation. A smaller RMSE means the fitted values are closer to the observed values.

Predicting New Rows

Supply a new data frame to the newdata argument when the response is not already observed. Its predictor column must have the same name used in the model.

new_players <- tibble(BA_2014 = c(0.241, 0.310, 0.265)) %>%
  mutate(predicted_BA_2015 = predict(batting_model, newdata = .))

new_players
## # A tibble: 3 × 2
##   BA_2014 predicted_BA_2015
##     <dbl>             <dbl>
## 1   0.241             0.258
## 2   0.31              0.291
## 3   0.265             0.270

These predictions still describe the fitted relationship learned from the 2014–2015 data. Lecture 7 will separate the data used to fit a model from the data used to evaluate its predictions.

How Correlation Determines the Line

If both variables are standardized, the correlation is the slope of the fitted line:

y−yˉsd(y)=r×x−xˉsd(x)\frac{y - \bar{y}}{sd(y)} = r \times \frac{x - \bar{x}}{sd(x)}

Rearranging gives the unstandardized slope and intercept:

b=r×sd(y)sd(x),a=yˉ−bxˉb = r \times \frac{sd(y)}{sd(x)}, \qquad a = \bar{y} - b\bar{x}

This explains why the sign of the slope always matches the sign of the correlation. In practice, lm() calculates the coefficients for us.