Can causality be established from this study? | Study design | AP Statistics | Khan Academy

A gym that specializes in weight loss offers its members an optional dietary program for an extra fee. To study the effectiveness of the dietary program, a manager at the gym takes a random sample of 50 members who participate in the dietary program and 50 members who don't. The manager finds that those who participated in the dietary program, on average, lost significantly more weight than those who didn't participate in the program in the past three months. The manager concludes that the dietary program caused the increased weight loss for the gym's members during that time period.

So pause this video and see and think about whether you think the manager is making a valid conclusion based on this study. All right, so let's first just make sure we understand what the manager is concluding. The manager is saying that there's a causal relationship between the dietary program; that being on the dietary program causes increased weight loss. They would be able to make this conclusion if it's a well-designed experimental study.

You might say, "Well, what does an experimental study look like?" Well, in an experimental study, you have a control group that wouldn't have the dietary program, and then you would have a treatment or experimental group that does have the dietary program. Then you would see, "Hey, if these folks are actually seeing more weight loss, and if it's statistically significant, then maybe you can conclude the causal relationship that the dietary program is causing the weight loss."

This kind of looks like that until you think about how folks were assigned to either of these groups. It would be a well-designed experimental study if you took random samples from the broader population. So, you took a random sample from the broader population, and then from this group, you randomly assign folks to either the control or the treatment.

So you're randomly assigning, and then the people who happened to be in the treatment group—not the people who chose to be in it—well, those people you'd say, "You need to be on this diet," and you make sure that they are on that diet. If you see a statistically significant increased weight loss, then you might be able to make this conclusion. But that's not what happened over here.

In this situation, we didn't randomly select from the broader population and then randomly assign folks to these groups. What happened is people self-selected into either being in the dietary program or not being in the dietary program—I'll say not the dietary program. Then what they did is that they randomly sampled from the dietary program to put them in, I guess you could say, the treatment group, and then they randomly sampled from this control group.

So in this situation, you have a significant confounding variable. People essentially self-selected themselves into the dietary program group, and that you could view as a confounding variable. It doesn't allow you to make this causal connection because people who selected themselves into the dietary program maybe are more motivated to lose weight. Maybe they could afford the money, and that wealth association is associated with being able to eat better and maybe being able to lose more weight.

Now a conclusion that the manager might be able to make based on this study is that there is an association between participating in the dietary program and weight loss. But once again, you can't establish the causality.