In this tutorial, you're going to learn about the principles of experimental design.
Experimental design simply means the way in which the experiment is carried out. Many experimental designs include a control group and a treatment group to compare effects of treatment (exercise, drug, video watching, etc). You can have a good design of an experiment or a poor design of an experiment.
Good designs will have these three components:
Control means holding everything else besides what you're trying to measure constant. The purpose is to determine whether or not your treatment is effective. In other words, if there is an observable difference between groups, is it due to the treatments or are they due to a confounding variable? It is important to control all other variables to help limit confounding.
Let’s say you’re a farmer and you want to try a new fertilizer in your field. One thing you could do is choose 10 fields with similar soil nutrients, sunlight, water; all these things that could affect the crop growth.
You could then apply the old fertilizer to five fields and the new fertilizer to the other five. And by keeping all the other variables-- soil nutrients, sunlight, water, and all these other things consistent, the differences between the fields can be isolated then to the old fertilizer or the new fertilizer.
Does the new fertilizer work? Is it effective? This is the idea behind controlling for all of these other variables.
The second big idea of experimental design is randomization. The treatments must be assigned to the subject using a random process, otherwise known as "randomization". The purpose of random assignment is to try and filter out all the other sources of variation that we couldn't think to control for.
Back to the farmer example, even though you made the fields as similar as possible with respect to water, sunlight, and soil, it's possible that there is a variable that you didn't think to control for.
Maybe some fields had moles under the ground, and that would affect how the crops grow. How would you know to control for moles? By randomly assigning treatments to the fields, you can hopefully get some fields with moles in fields with the new and old fertilizer. Randomization smooths out those effects other variables might bring into the equation.
Randomizing also helps avoid bias, because you can’t be tempted to assign treatments to the experimental units you think might give favorable outcomes.
Randomization in an experiment is not really the same purpose as random selection in a sample. When you do a simple random sample, the idea is to get a sample that's representative of the population. In an experiment, the purpose of randomly assigning individuals to groups is to filter out unknown sources of variation.
The assignment in an experiment is pretty similar to the way you would randomly select in a sample.
Replication is the last key idea in experimental design, which says that a bigger sample is better, basically. Repeating the experiment on multiple subjects or experimental units is a better idea than doing few. Why is that? Well, a larger size of the experiment means it's more likely that we can find trends that maybe we wouldn't have found in a smaller experiment.
What if you had only two fields that were similar to each other, instead of 10? Let’s say you randomly assigned one to get the new fertilizer and one to get the old.
Isn't it possible in that that maybe the field with the old fertilizer does very well just by random chance? And that makes it seem like the new fertilizer is not effective when maybe it is. Or the opposite could happen, where it seems like the fertilizer is effective when it's not.
By randomly assigning five plots, it's more likely that you would find trends among those five plots that you can trust more than if you had just done one.
The more you replicate, and the more experimental units used in your experiment, brings about a more likely occurrence that you'll find the true trends that arise, rather than some freak anomaly.
The components of an experimental design, that is, a well-designed experiment are control, randomization, and replication. Control, again, helps to isolate the effects of the treatments, randomization helps to make the groups as similar as we can and helps to avoid bias, and then replication helps us to see the differences that might not have been evident if we had used a small sample. And so we've talked about treatments, which are the things that you apply to your experimental units, and then control, randomization, and replication.
Source: This work is adapted from Sophia author Jonathan Osters.
The way in which an experiment is carried out. A good design has key elements of randomization, replication, and control.
Something the researchers administer to the subjects or experimental units.
The principle of experimental design that requires that other variables which may confound the experiment be held constant between the treatment groups, so that any differences in the groups can be attributed to the different treatments.
The principle of experimental design that requires that the subjects/experimental units be assigned to groups using some random process. This ensures that the two groups are roughly equal prior to assigning treatments.
Repeating the experiment on multiple subjects/experimental units. This principle of experimental design that states that a larger experiment with more subjects/experimental units will allow us to more clearly see differences between the treatments.