yego.me
💡 Stop wasting time. Read Youtube instead of watch. Download Chrome Extension

Standard deviation of residuals or Root-mean-square error (RMSD)


4m read
·Nov 11, 2024

What we're going to do in this video is calculate a typical measure of how well the actual data points agree with a model—in this case, a linear model. There are several names for it; we could consider this to be the standard deviation of the residuals, and that's essentially what we're going to calculate. You could also call it the root mean square error, and you'll see why it's called this because this really describes how we calculate it.

So what we're going to do is look at the residuals for each of these points, and then we're going to find the standard deviation of them. Just as a bit of review, the ith residual is going to be equal to the y-value for a given x minus the predicted y value for a given x. Now, when I say y-hat right over here, this just says what would the linear regression predict for a given x, and this is the actual y for given x.

So for example—and we've done this in other videos—this is all review. The residual here, when x is equal to 1, we have y equal to 1, but what was predicted by the model is 2.5 times 1 minus 2, which is 0.5. So, 1 minus 0.5; this residual here is equal to 1 minus 0.5, which is equal to 0.5. It's a positive 0.5, and if the actual point is above the model, you're going to have a positive residual.

Now, the residual over here, you also have the actual point being higher than the model, so this is also going to be a positive residual. Once again, when x is equal to 3, the actual y is 6. The predicted y is 2.5 times 3, which is 7.5 minus 2, which is 5.5. So you have 6 minus 5.5; here, I'll write residual is equal to 6 minus 5.5, which is equal to 0.5. So once again, you have a positive residual.

Now for this point that sits right on the model, the actual is the predicted. When x is 2, the actual is 3, and what was predicted by the model is 3. So the residual here is equal to the actual is 3, and the predicted is 3, so it's equal to zero. Last but not least, you have this data point where the residual is going to be the actual. When x is equal to 2, it is 2 minus the predicted.

Well, when x is equal to 2, you have 2.5 times 2, which is equal to 5 minus 2, which is equal to 3. So 2 minus 3 is equal to negative 1. When your actual is below your regression line, you're going to have a negative residual, so this is going to be negative 1 right over there.

Now we can calculate the standard deviation of the residuals. We're going to take this first residual, which is 0.5, and we're going to square it. We're going to add it to the second residual right over here. I'll use this blue with this teal color: that's zero; I'm gonna square that. Then we have this third residual, which is negative 1, so plus negative 1 squared. Finally, we have that fourth residual, which is 0.5 squared; 0.5 squared.

So once again, we took each of the residuals—which you could view as the distance between the points and what the model would predict—we are squaring them. When you take a typical standard deviation, you're taking the distance between a point and the mean. Here, we're taking the distance between a point and what the model would have predicted, but we're squaring each of those residuals and adding them all up together.

Just like we do with the sample standard deviation, we are now going to divide by one less than the number of residuals we just squared and added. We have four residuals; we're gonna divide by four minus one, which is equal to, of course, three. You could view this part as a mean of the squared errors, and now we're going to take the square root of it.

So let's see, this is going to be equal to the square root of—this is 0.25, 0.25—this is just 0—this is going to be positive 1—and then this 0.5 squared is going to be 0.25, 0.25—all of that over 3. Now, this numerator is going to be 1.5 over 3. So this is going to be equal to 1.5, which is exactly half of three, so we could say this is equal to the square root of one half.

This is one over the square root of two; one divided by the square root of two, which gets us 2. If we round to the nearest thousandths, it's roughly 0.707, so approximately 0.707. If you wanted to visualize that, one standard deviation of the residuals below the line would look like this, and one standard deviation above the line for any given x value would go one standard deviation of the residuals above it. It would look something like that, and this is obviously just a hand-drawn approximation.

But you do see that this does seem to be roughly indicative of the typical residual. Now, it's worth noting sometimes people will say it's the average residual, and it depends on how you think about the word average because we are squaring the residuals. So outliers—things that are really far from the line—when you square it are going to have a disproportionate impact.

If you didn't want to have that behavior, we could have done something like find the mean of the absolute residuals. That actually, in some ways, would have been a simpler one, but this is a standard way of people trying to figure out how much a model disagrees with the actual data. You can imagine the lower this number is, the better the fit of the model.

More Articles

View All
The Stock Market Is Broken
What’s up, you guys? It’s Graham here. So, we got to take a moment and talk about something serious—something that no investor wants to think about, something that’s happened in the past but many people feel is going to happen again in the future. And no,…
Saturn 101 | National Geographic
[Instructor] With its gold color and stunning rings, Saturn is quite a planetary gem. Saturn is the second-largest of the eight planets, and it is about ten times as wide as Earth. Despite its size, Saturn is actually the lightest planet. It is predomin…
Death by Black Hole
As of lately, it seems that everybody is trying to tell you when and how the world will end. Some scenarios are far more familiar and likely than others. Those that are widely discussed in the media range from infectious diseases to nuclear war, all the w…
More Lies About the World You Believe
So you’re 11 years old. You’ve just scarfed down some mac and cheese and birthday cake. You and your friends run wildly, eager to jump in the pristine blue pool on a hot summer day. And then your mom stops you, saying, “No swimming yet! Wait 30 minutes!” …
"He Saved My Life" American Soldier Returns to Help Iraqi Captain Fleeing ISIS | National Geographic
[Music] [Music] Ian yes for [Music] I’m very scared to lose my son, lose my daughter, lose my wife, thus all my [Music] life. The soldiers, like the captain, are the ones that kept us alive. My name is Chase Msab. I’m a veteran of the Iraq War. I did thre…
How to Invest for Beginners (2022)
All right, here we go! Welcome, guys! In this video, I’m going to be doing a full beginner’s guide to investing in the stock market. So buckle up! If you’re a beginner, you want to invest but you’ve never bought a stock before, then this video is definite…