Worked example identifying observational study | Study design | AP Statistics | Khan Academy

So we have a type of statistical study described here. I encourage you to pause this video, read it, and see if you can figure out: Is this a sample study? Is it an observational study? Is it an experiment? And then also think about what type of conclusions can you make based on the information in the study.

All right, now let's work on this together. British researchers were interested in the relationship between farmers' approach to their cows and cow's milk yield. They prepared a survey questionnaire regarding the farmers' perception of the cow's mental capacity, the treatment they give to the cows, and the cow's yield. The survey was filled by all the farms in Great Britain. After analyzing their results, they found that on farms where cows were called by name, milk yield was 258 liters higher on average than on farms when this was not the case.

All right, so they're making a connection between two variables. One was whether cows were called by name—whether cows named. All right, whether cows named. And this would be a categorical variable because for any given farmer, it's going to be a yes or no that the cows are named. So they're trying to form a connection between whether the cows are named and milk yield. This would be a quantitative variable because you're measuring it in terms of the number of liters—milk yield.

Whether we are drawing a connection, and they're able to draw some form of a connection, they're saying, “Hey, when the cows were called by name, milk yield was 258 liters higher on average than on farms when this was not the case.”

So first, let's just think about what type of statistical study this is. We could think, okay, is this a sample study? Is this a sample study? Is this an observational study—observational—or is this an experiment? Now, a sample study—an experiment—a sample study you would be trying to estimate a parameter for a broader population. Here, it's not so much that they're estimating the parameter; they're trying to see the connection between two variables.

That brings us to observational study because that's what an observational study is all about: can we draw a connection? Can we draw a positive or negative correlation between variables based on observations? So we've surveyed a population here—the farmers in Great Britain—and we are able to draw some type of connection between these variables. This is clearly an observational study.

Now, this is not an experiment. If there was an experiment, we would take the farmers and we would randomly assign them into one or two groups. In one group, we would say, “Don't name—no naming!” In the other group, we would say, “Name your cows!” Then we would wait some period of time and we will see the average milk production going into the experiment in the no-naming group and the naming group. Next, we would wait some period of time—six months, a year—and then we’ll see the average milk production after either not naming or naming the cows for six months.

So that's not what occurred here. Here, we just did the survey to everybody. We just asked them this question, and we were able to find this connection between whether the cows were named and the actual milk yield. So clearly, not an experiment; this was an observational study.

Now, the next thing is, what can we conclude here? We know when— you know, they’re telling us that when the cows were named, it looks like there was a 258-liter higher yield on average. So the conclusion that we can strictly make here is: well, for farmers in Great Britain, there is a correlation, a positive correlation between whether cows are named and the milk yield. So that we can say for sure.

So let me write that down. For Great Britain farmers, we have a positive correlation between naming cows and milk yield. That's pretty much what we can say here.

Now, some people might be tempted to try to draw causality. You’ll see this all the time where you see these observational studies, and people try to hint, “Maybe there’s a causal relationship here. Maybe the naming is actually what makes the milk yield go up.” Or maybe it's the other way—the cows produce a lot of milk, the farmers like them more, and they want to name them because, like, “Hey, that's my high milk-producing cow.”

So there's a lot of temptation to say, “Naming—that maybe there's a cause out of that; naming causes more milk,” or that maybe more milk causes naming. The farmers really like that cow, so they start naming them. Or whatever it might be. But you can't make this causal relationship based on this observational study. You might have been able to do it with a well-constructed experiment, but not with an observational study.

That's because there could be some confounding variable that is driving both of them. For example, that confounding variable might just be a nice farmer. And, you know, we can define nice in a lot of ways—they're gentle. A nice farmer is more likely to name cows, and a nice farmer is more likely to get a higher yield. The reason why this is a confounding variable is: if you were to control for that—if you just take, “Well, let's just control for nice farmers and then see if naming makes a difference,” it might not make a difference.

If the farmer is, you know, petting the cows and treating them humanely and doing other things, it might not matter whether the farmer names them or not. Likewise, if you take some less nice farmers who, you know, hit their cows and they have really inhumane conditions, it might not make a difference whether they name the cows or not.

So it's very important that you—from the observational studies—you might—if they’re well-constructed—you might be able to say there's a correlation. But you won't be able to make a causal conclusion.