Why residual plots

Topics: Regression Analysis. Anyone who has performed ordinary least squares OLS regression analysis knows that you need to check the residual plots in order to validate your model. Have you ever wondered why? The bottom line is that randomness and unpredictability are crucial components of any regression model. This is the part that is explained by the predictor variables in the model.

The expected value of the response is a function of a set of predictor variables. Stochastic is a fancy word that means random and unpredictable. Error is the difference between the expected value and the observed value. Putting this together, the differences between the expected and observed values must be unpredictable. The idea is that the deterministic portion of your model is so good at explaining or predicting the response that only the inherent randomness of any real-world phenomenon remains leftover for the error portion.

If you observe explanatory or predictive power in the error, you know that your predictors are missing some of the predictive information.

Residual plots help you check this! Statistical caveat: Regression residuals are actually estimates of the true error, just like the regression coefficients are estimates of the true population coefficients. Using residual plots, you can assess whether the observed error residuals is consistent with stochastic error.

Since this is a form of error, the same general assumptions apply to the group of residuals that we typically use for errors in general: one expects them to be roughly normal and approximately independently distributed with a mean of 0 and some constant variance. This means that an analyst should expect a regression model to err in predicting a response in a random fashion; the model should predict values higher than actual and lower than actual with equal probability.

In addition, the level of the error should be independent of when the observation occurred in the study, or the size of the observation being predicted, or even the factor settings involved in making the prediction.

The overall pattern of the residuals should be similar to the bell-shaped pattern observed when plotting a histogram of normally distributed data.

We emphasize the use of graphical methods to examine residuals. Departures from these assumptions usually mean that the residuals contain structure that is not accounted for in the model. Identifying that structure and adding term s representing it to the original model leads to a better model. Any graph suitable for displaying the distribution of a set of data is suitable for judging the normality of the distribution of a group of residuals.

The three most common types are: histograms , normal probability plots , and dot plots. The histogram is a frequency plot obtained by placing the data in regularly spaced cells and plotting each cell frequency versus the center of the cell. Figure 2. We have superimposed a normal density function on the histogram. A more sensitive graph is the normal probability plot. The steps in forming a normal probability plot are: Sort the residuals into ascending order.

Plot the calculated p -values versus the residual value on normal probability paper. Each data point has one residual. A residual plot is a graph that shows the residuals on the vertical axis and the independent variable on the horizontal axis. If the points in a residual plot are randomly dispersed around the horizontal axis, this means that our linear regression model is appropriate for the data; otherwise, a non-linear model is more appropriate.

Example of a residual plot showing that our linear regression model is the best fit to our data. And let's see, the maximum residual here is positive. So let's see, this could be. So this is negative one. This is positive one here. And so when x equals one, what was the residual? Well, the actual was one, expected was 0. So this right over here, we can plot right over here. The residual is 0. When x equals two, we actually have two data points.

First, I'll do this one. When we have the point two comma three, the residual there is zero. So for one of them, the residual is zero. Now for the other one, the residual is negative one. Let me do that in a different color. For the other one, the residual is negative one, so we would plot it right over here. And then this last point, the residual is positive. So it is just like that. And so this thing that I have just created, where we're just seeing, for each x where we have a corresponding point, we plot the point above or below the line based on the residual.