Impact of mass on orbital speed | Study design | AP Statistics | Khan Academy

Let's say that we've come up with a new pill that we think has a good chance of helping people with diabetes control their blood sugar. When someone has diabetes, their blood sugar is unusually high, which damages their body in a bunch of different ways. So, we want to conduct an experiment to test if this pill really can help people lower their blood sugar.

The first thing we need to think about is how do we even measure or test whether people's blood sugar is getting lowered? Well, for our experiment, what's typically done is we measure folks' hemoglobin A1c. You don't have to worry too much about this in the context of statistics, but a hemoglobin A1c test is a way that's typically used to measure your average blood sugar over the last three months. We have whole videos on Khan Academy explaining how that works.

Our hope would be that our pill lowers people's blood sugar, which shows up as a lowered A1c. Now we have terms for this: the thing that is causing something else to change, we call this the explanatory variable. The explanatory variable is the thing that might get changed by that explanatory variable depending on whether you take the pill or not. We call that our response variable.

So now, let's actually conduct the experiment. What we would do is we would go to the population of diabetics, and we would want to take a random sample from that population of diabetics, a reasonably large one. Later in statistics, we talk about what a good size sample might be. Let's say that we randomly sample 100 folks.

We randomly sample 100 folks from that population of diabetics, and then you would want to assign these folks randomly to two different groups. One would be your control group, and this would be the group of people who won't take the new medicine. Then you would have your treatment group; these are the groups of folks who will be given the new medicine.

Now, in some cases, you can just randomly assign these hundred folks between these two groups. One way to do it is you could give all of them a random number between one and a hundred. The top 50 go into treatment, and the bottom 50 go into the control, or others, you could use a computer to randomly assign folks.

Sometimes, you might want to be a little bit more sophisticated than that. For example, there might be evidence that someone's sex might somehow influence how they respond to a drug. So, what you could do is something called block design, where, let's say this group just happens to have 60 females and 40 males. In one block design, you can randomly assign, but you can do it in a way that ensures both of these groups have the same proportions of males and females.

For example, if you have 60 females here, you can ensure that 30 of them end up in the control and 30 end up in the treatment group. But you would assign those 60 females randomly between these two groups. Similarly, you can do a block design for these 40 males, with 20 ending up in the control and 20 ending up in the treatment.

Once you have folks in both of these groups, what you would probably want to do is measure their A1c at the beginning. You could view that as a baseline, and then over the course of the experiment, you would give the pill to the treatment group. In the control group, you might be saying, well, we just wouldn't do anything. But the best practice is actually to give a pill that looks just like the real thing to the control group. This is known as a placebo.

The reason why we do that is there's definitely evidence that when people think they're taking a pill that might help them, even psychologically, it can have an effect on them, and sometimes it helps them. This is known as the placebo effect. Not only would you give both groups a pill that looks the same, even though this one in the treatment group actually has the medicine in it, you also would not want to tell folks which group they are in.

When you don't tell them which group they're in, that's known as a blind experiment. You probably also don't want to tell the people who are administering the experiment which group they are administering. That's called a double-blind. So even the doctors or the nurses that are administering the experiment, when they're giving a pill to the control group, they don't know that that pill is the placebo.

You might say, well, why is it important for an experiment to be blind or especially double-blind? Well, that avoids any type of psychological effect from the point of the patient or from, say, the caregivers in this situation. They don't kind of give it away; they don't tell these folks, "Hey, you're actually just pretending to take a pill."

That ensures that we minimize the amount of influence or bias that might happen. You might even have a triple-blind experiment where even the folks who are analyzing the eventual data from this experiment don't know whether they're analyzing the data from the control or the treatment. They just compare the two different groups.

But anyway, people take the medicine and the placebo over the course of the experiment, and maybe this lasts for three months. Then you would want to measure their A1c later, and then you would see their change in the A1c.

Now, if you saw that there wasn't really a difference in the change in A1c between the control and the treatment group, then you'd say, well, that probably means that my pill didn't work. Now, if you do get a greater reduction in the treatment group and you do the statistical analysis, which we will learn in statistics, and you show that, hey, there’s a very low probability this happened purely due to chance, well then you've got something.

You could probably conclude that there is a causal connection between taking the pill and lowering your A1c level. But once again, you cannot be 100% sure, and so this is why it's very important for people to be able to replicate your experiment. What you'd want to do, either yourself or other researchers, might want to conduct the experiment with different sample sizes, in different countries, and different populations, maybe with different ages at different times of the year to ensure that they continue to see this result.