yego.me
💡 Stop wasting time. Read Youtube instead of watch. Download Chrome Extension

Introduction to residuals and least-squares regression | AP Statistics | Khan Academy


3m read
·Nov 11, 2024

Let's say we're trying to understand the relationship between people's height and their weight. So what we do is we go to 10 different people and we measure each of their heights and each of their weights.

And so on. This scatter plot here, each dot represents a person. So, for example, this dot over here represents a person whose height was 60 inches or 5 feet tall. So that's the point (60, ) and whose weight, which we have on the y-axis, was 125 pounds.

And so when you look at this scatter plot, your eyes naturally see some type of a trend. It seems like, generally speaking, as height increases, weight increases as well. But I said, generally speaking, you definitely have circumstances where there are taller people who might weigh less.

But an interesting question is, can we try to fit a line to this data? This idea of trying to fit a line as closely as possible to as many of the points as possible is known as linear regression. Now, the most common technique is to try to fit a line that minimizes the squared distance to each of those points.

And we're going to talk more about that in future videos, but for now, we want to get an intuitive feel for that. So if you were to just eyeball it and look at a line like that, you wouldn't think that it would be a particularly good fit. It looks like most of the data sits above the line.

Similarly, something like this also doesn't look that great. Here, most of our data points are sitting below the line. But something like this actually looks very good. It looks like it's getting as close as possible to as many of the points as possible.

It seems like it's describing this general trend, and so this is the actual regression line. The equation here we would write as, and we'd write y with a little hat over it, and that means that we're trying to estimate a y for a given x.

It's not always going to be the actual y for a given x because, as we see, sometimes the points aren't sitting on the line. But we say y hat is equal to, and our y-intercept for this particular regression line is negative 140 plus the slope 14 over 3 times x.

Now, as we can see, for most of these points, given the x value of those points, the estimate that our regression line gives is different than the actual value. And that difference between the actual and the estimate from the regression line is known as the residual.

So let me write that down. So, for example, the residual at that point is going to be equal to, for a given x, the actual y value minus the estimated y value from the regression line for that same x.

Or another way to think about it is, for that x value when x is equal to 60, we're talking about the residual just at that point. It's going to be the actual y value minus our estimate of what the y value is from this regression line for that x value.

So pause this video and see if you can calculate this residual, and you could visually imagine it as being this right over here. Well, to actually calculate the residual, you would take our actual value, which is 125 for that x value.

Remember, we're calculating the residual for a point, so it's the actual y there minus what would be the estimated y there for that x value. Well, we could just go to this equation and say what would y hat be when x is equal to 60?

What's it going to be equal to? Let's see. We have negative 140 plus 14 over 3 times 60. Let's see, 60 divided by 3 is 20. 20 times 14 is 280. And so all of this is going to be 140.

And so our residual for this point is going to be 125 minus 140, which is negative 15. And residuals indeed can be negative. If your residual is negative, it means for that x value, your data point, your actual y value is below the estimate.

If we were to calculate the residual here or if we were to calculate the original here, our actual for that x value is above our estimate, so we would get positive residuals. And as you will see later in your statistics career, the way that we calculate these regression lines is all about minimizing the square of these residuals.

More Articles

View All
How to Get and Evaluate Startup Ideas | Startup School
[Music] All right, hello everyone! I’ve got a lot of content to get through, so I’m gonna move fast. Buckle in! If you are looking for a startup idea right now, I’m going to try to help. But more importantly, I’m going to try to give you the conceptual t…
We’re at the Beginning of an Infinity of Knowledge
The difference with “The Beginning of Infinity” is that you’re getting a worldview. You’re not being given the standard take from physicists about how to understand quantum theory. You’re not being given the standard take of how to understand knowledge fr…
Experiencing the Rich History and Food of Canada's Season of Bounty | National Geographic
Iconic! I love to know the life of the stuff I’m going to eat before it shows up on my plate. That’s why I’m in Canada’s Maritime Provinces and Quebec during the season of bounty. To connect with the land and rich history through people and food. Good mo…
The Fourth Amendment | Civil liberties and civil rights | US government and civics | Khan Academy
Hey, this is Kim from Khan Academy, and today I’m talking with some experts about the 4th Amendment. This is the 4th Amendment of the Bill of Rights, and the 4th Amendment deals with unreasonable search and seizure. So, here’s the official text of the am…
The 5 Step Process for Getting What You Want From Life
Like I say, you can have practically anything you want in life, but you can’t have everything you want in life. So that means you have to prioritize what are the things you’re going after that has to do with the earlier part of, you know, knowing what you…
World's Heaviest Weight
An apple weighs about 1 newton; the world record for jet engine thrust is 570,000 newtons. And the Saturn V rocket that launched people to the moon had a thrust of 33,360,000 newtons. But how can we measure forces this big accurately? Well, we need to ask…