Matched pairs experiment design | Study design | AP Statistics | Khan Academy

The last video, we constructed an experiment where we had a drug that we thought might help control people's blood sugar. We looked for something that we could measure as an indicator of whether blood sugar is being controlled, and hemoglobin A1c is actually what people measure in a blood test. We have a whole video on it on KH Academy, but it is an average measure of your blood sugar over roughly a 3-month period.

So that's the explanatory variable: whether or not you're taking the pill. The response variable is, well, what does it do to your hemoglobin A1c? We constructed a somewhat classic experiment where we had a control group and a treatment group. We randomly assigned folks into either the control or the treatment group, and to ensure that one group or the other, or I guess both of them, don't end up with an imbalance—in the case of the last video, an imbalance of men or women—we did what we call a block design.

We took our 100 people, and we just happened to have 60 women and 40 men. We said, okay, well, let's split the 60 women randomly between the two groups and let's split the 40 men between these two groups, so that we have at least an even distribution with respect to sex. We would measure folks’ A1c’s before they get the treatment or the placebo. Then we would wait three months of getting either the treatment or the placebo, and then we'll see if there's a statistically significant improvement.

Now, this was a pretty good, and it's a bit of a classic experimental design. We would also do it so that the patients don’t know which one they’re getting: placebo or the actual treatment. So, it's a blind experiment, and it would probably be good if even the nurses or the doctors who are administering the pills, who are giving the pills, also don’t know which one they’re giving. So, it would be a double-blind experiment.

But this doesn’t mean that it’s a perfect experiment, and there seldom is a perfect experiment. That’s why it should be able to be replicated; other people should try to prove the same thing, maybe in different ways. But even the way that we designed it, there’s still a possibility that there are some lurking variables in here. Maybe we took care to make sure that our distribution of men and women was roughly even across both of these groups, but maybe through that random sampling, we got a disproportionate number of young people in the treatment group.

And maybe young people responded better to taking a pill. Maybe that, you know, it changes their behaviors in other ways, or maybe older people, when they take a pill, they decide to eat worse because they say, “Oh, this pill is going to solve all my problems.” And so, you could have these other lurking variables like age, or where in the country they live, or other types of things that just by the random process might get uneven in one way or another.

Now, one technique to help control for this a little bit—I shouldn’t use the word control too much—another technique to help mitigate this is something called matched pairs design. Matched pairs design of an experiment is essentially, instead of going through all of this trouble, saying, “Oh boy, maybe we do block design, all this random sampling,” instead, you randomly put people first into either the control or the treatment group.

Then we do another round; you measure, and then you do another round where you switch. The people who are in the treatment go to the control, and the people who are in the control go into the treatment. So, we could even extend from what we have here. We can imagine a world where the first three months, we have 50 people in this treatment group. We have another 50 people in this control group that are taking the placebo.

We see what happens to the A1c’s, and then we switch, where this group over here, then—and they don’t know—they don’t know, first of all, ideally it’s a blind experiment, so they don’t even know they were in the treatment groups. Hopefully, the pills look identical. So now that same group for the next three months is now going to be the control group.

They got the medicine for the first three months, and we saw what happens to their A1c. Now they’re going to get the placebo for the second three months, and then we are going to see what happens to their A1c. Likewise, the other group is going to be switched around: the folks that used to be getting the placebo could now get the treatment. They are now going to get the treatment.

The value here is that because everyone is going through both—being in the control group for one period and in the treatment group for another—and they don’t know when which one is happening, you are less likely to have a lurking variable, like age or geographic region or behavior, cause an imbalance or somehow skew the results or give you biased results.

So, this is an interesting thing, and you know, even what I’ve talked about in this video and the last one, these are just different ways to approach it. As you construct experiments, this is in medicine; you’ll obviously construct experiments in other fields. It’s important to think about what types of things are practical to do and also have the best chance at giving you, I guess you could say, real, unbiased information as to, in the case of an experiment, the efficacy of something or whether a certain explanatory variable really does drive a causal effect on a response variable.