Use Sophia to knock out your gen-ed requirements quickly and affordably. Learn more
×

Issues with Performing Experiments

Author: Sophia

what's covered
This lesson discusses issues with performing experiments. By the end of the lesson, you will understand what bias is and how it can affect the outcome of an experiment. You will also learn what extraneous variables are and how they can impact an experiment. This lesson covers:

Table of Contents

1. Bias

Think back to a previous lesson where you learned about the experimental method. Specifically think back to the fourth step, the one where you test the predictions regarding how variables interact and record data.

  • Step 4: Test or experiment with the prediction by trying to determine if, in fact, the things we are looking at share the cause-and-effect relationship that was predicted.
In statistics, there are populations and samples. Experiments use samples in an effort to learn more about a population. Two typical problems that occur when conducting experiments can impact the results of that experiment. One problem is what is referred to as bias, and the other problem is associated with not accounting for all variables, specifically those referred to as extraneous variables.

Bias is the result of something in the experiment causing measurements that are not representative of the population. Bias causes the parameters of a population to be misrepresented as the parameter is typically over- or underestimated. Because of bias, data collected from an experiment cannot be applied to a greater group of things; that data is not reliable.

IN CONTEXT

Say a group of friends or coworkers decides to see who can lose the most weight in a month. In a case like this, bias would exist if the scale were not accurately measuring the weight of those who use it. If the scale hasn’t been calibrated and it’s off, that could have an impact on the results of the weight loss contest. While conducting the experiment, any error resulting from the way the experiment is designed is known as bias.

Unless this bias is addressed, the error will still exist if the experiment is repeated. People keep getting on the same scale over and over again. It’s going to be inaccurate every single time. Therein lies the bias.

Even if an experiment does not have any bias, the results can still be impacted by random errors. In the case of the weight loss contest, a random error could be the case of somebody simply misreading the scale and writing down the wrong weight.

terms to know
Bias
An error caused by the experimental design creating a situation where the values related to parameters of a population will consistently be over- or underestimated.
Extraneous Variables
Variables that are not controlled in the experimental design.


2. Extraneous Variables

The goal of the experimental method is to determine whether or not there is a cause-and-effect relationship between variables. In addition to any possible bias encountered, all variables that might affect the response variable being studied must be accounted for.

Those variables that are not accounted for are called extraneous variables. Extraneous variables can cause a misinterpretation of the cause-and-effect relationship that may exist between two variables.

IN CONTEXT

Lars goes to the doctor’s office and complains about certain symptoms that might be indicative of a particular condition. The doctor prescribes a drug and asks Lars to come back in a couple of weeks. At his follow-up appointment, Lars’s symptoms are gone.

While the prescribed drug may have worked, it’s possible that the condition was an allergic reaction to another medicine. If Lars stopped taking that initial medicine, well, this might be a case of an extraneous variable. It wasn’t the medicine the doctor prescribed that caused the symptoms to go away. It was related to something else.


3. Selection Bias

When conducting an experiment, there are two types of bias that require special attention: selection bias and participation bias. Selection bias can occur when the random assignment of individuals or things into groups is not done or is not possible.

Random assignment is done so you don’t over- or underrepresent a specific portion of the population. You randomly pick people or observations to try to avoid having all the observations with the same characteristics.

IN CONTEXT

A good example of selection bias would be a telephone poll where a poller or an experimenter decides to call people and ask them questions about a particular issue. The experimenter picks the landline numbers based on public record. The experimenter is specifically underrepresenting all people who might have a telephone by missing people that don't use a landline.

The selection bias would be only choosing people that have a particular type of telephone number, a landline, which underrepresents the population. It’s going to introduce bias into the results here. Observations that are left out of the study like this may cause a study to not represent the larger group of individuals being studied.


4. Participation Bias

Participation bias is a special type of selection bias that can occur when participants have a choice of whether to participate in an experiment. Those who choose to participate may have very different characteristics from those who choose not to, and that can influence the results of an experiment.

IN CONTEXT

Say your shopping receipt has a code at the bottom, and the salesperson tells you, “If you type in this website and answer the survey questions, you’ll be entered to win a $5,000 gift card.” The possibility of winning the gift card may prompt somebody to participate in the survey, and participation that is incentivized in this way creates participation bias. Participation in the survey (the experiment) is entirely up to the individual.

big idea
Selection bias is associated more as the result of an error by the researcher in selecting participants whereas participation bias is the result of the choices made by those participants. Since there are a lot of possible sources of bias in an experiment, it is extremely important for a researcher to be aware of them.
  • Selection Bias: Related to researcher error.
  • Participation Bias: Related to participant choices.


5. Bias and Extraneous Variables

There are multiple steps in the experimental method process. In Steps 2, 6, and 7 of the process, whether we are coming up with a hypothesis, drawing a conclusion about an experiment, or simply trying to revise our hypothesis, taking into consideration bias and its effect on the experiment leads to stronger conclusions.

  • Step 2: Make a guess as to how these elements might have a cause-and-effect relationship.
  • Step 6: Conclude whether or not the test showed that the prediction was correct or incorrect.
  • Step 7: Revise your guess if the prediction was wrong and start from Step 2. -or- If the prediction looks plausible, start testing again from Step 4 to verify your results.
Multiple variables can impact the response variable being studied in an experiment. Typically, errors in an experiment can be the result of variables that are not considered by the researcher. These are called extraneous variables.

When selection or participation bias occurs, the groups being studied might have features that are impacted by an extraneous variable that was not considered. Random assignment reduces the effects of extraneous variables since fewer participants with the same characteristics are assigned to a specific group.

When there is no random assignment and many participants have the same characteristics, the extraneous variable may impact the group being tested in the same way and alter the results of the data.

IN CONTEXT

To illustrate this idea, we’ll use an SAT prep course. The extraneous variable is how much time the students spend studying outside of class. It is very possible an experimenter could draw a false conclusion based on the fact that the students simply took the class to begin with and did not consider how much time they spend studying outside of the class.

When reading statistical results, it is important to look for such extraneous variables since they can be responsible for misleading or even incorrect results. Therefore, you need to take them into consideration when working in the experimental method process, much like you take bias into consideration.

summary
In this lesson, you learned what bias is and how it can affect the outcome of an experiment. You also learned what an extraneous variable is and how it can also impact an experiment. There are two key types of bias: selection bias and participation bias. There is a relationship between bias and extraneous variables. A bias can lead to an extraneous variable, which is one that is not being considered in the experiment.

Source: THIS TUTORIAL WAS AUTHORED BY DAN LAUB FOR SOPHIA LEARNING. PLEASE SEE OUR TERMS OF USE.

Terms to Know
Bias

An error caused by the experimental design creating a situation where the values related to parameters of a population will consistently be over- or underestimated.

Extraneous Variable

Variables that are not controlled in the experimental design.