Online College Courses for Credit

2 Tutorials that teach Issues with Performing Experiments
Take your pick:
Issues with Performing Experiments

Issues with Performing Experiments

Author: Dan Laub

In this lesson, students will learn about some common obstacles that can arise when performing an experiment.

See More

Try Our College Algebra Course. For FREE.

Sophia’s self-paced online courses are a great way to save time and money as you earn credits eligible for transfer to many different colleges and universities.*

Begin Free Trial
No credit card required

29 Sophia partners guarantee credit transfer.

312 Institutions have accepted or given pre-approval for credit transfer.

* The American Council on Education's College Credit Recommendation Service (ACE Credit®) has evaluated and recommended college credit for 27 of Sophia’s online courses. Many different colleges and universities consider ACE CREDIT recommendations in determining the applicability to their course and degree programs.


Source: Image of weightlifter, public domain,; Image of scale, public domain,; Image of stethoscope, public domain,; Image of pills, public domain,; Image of phone, public domain,; Image of question mark, public domain,; Image of package, public domain,; Image of teacher, public domain,

Video Transcription

Download PDF

[MUSIC PLAYING] Hi. Dan Laub here. And in this lesson, we're going to discuss issues with performing experiments. But before we get started, let's talk about a few of the objectives for this lesson.

The first objective is to understand what bias is and how it can affect the outcome of an experiment. And so we're going to discuss what bias is, we're going to define bias, and we're going to talk about the impact that it can have on a variety of experiments. And the second is to know what extraneous variables are and how they can impact an experiment, and we're going to discuss those as well. So let's get started.

So take a minute and think back to a previous lesson where we talked about the experimental method. What I want to discuss initially in this lesson is the fourth step, and the fourth step is the one where we test the predictions regarding how variables interact and record data. And so the step four is as follows. "Test or experiment with the prediction by trying to determine if, in fact, the things we are looking at show the cause and effect relationship that was predicted."

And so in statistics, there are populations and samples. And experiments use samples in an effort to try and learn more about a population. In some cases, there might be two typical problems that occur when conducting experiments.

These can impact the results of an experiment. One problem is what is referred to as "bias," and the other problem is associated with not accounting for all variables. The variables that are referred to as "extraneous variables."

Bias is the result of something in the experiment causing measurements that are not representative of the population. Bias causes the parameters of a population to be misrepresented as the parameter is typically over- or underestimated. Because of bias, data collected from an experiment cannot be applied to a greater group of things-- meaning that the data is not reliable.

So let's look at an example. In this example, let's consider a weight loss contest. So let's say a group of friends or coworkers decides to see who can lose the most weight in a month. And in a case like this, bias would exist if the scale is not accurately measuring the weight of those who use it.

So what if the scale hasn't been calibrated, and it's off by a pound or two? Well, obviously, that could have an impact on the results of the weight loss contest. While conducting the experiment, any error resulting from the way that the experiment is designed is known as "bias." Unless this bias is addressed, the error will still exist if the experiment is repeated.

So in this case, people keep getting on the same scale over and over again. It's going to be wrong every single time. Therein lies the bias.

Even if an experiment does not have any bias, the results can still be impacted by random errors. In the case of the weight loss contest, a random error could be the case of somebody simply misreading the scale and they maybe wrote down the wrong weight.

Remember, the gold experimental method is to determine whether or not there is a cause and effect relationship between variables. In addition to any bias it has encountered when an experiment is conducted, all variables that might affect the response variable being studied must be accounted for. Those variables that are not accounted for are called "extraneous variables." Extraneous variables can cause a misinterpretation of the cause and effect relationship that may exist between two variables.

So as another example, let's say we have a patient that goes to the doctor's office, and they complain about certain symptoms that might be indicative of a particular condition. So the doctor prescribes a drug, asks them to come back in a couple weeks, and the patient returns. And at the point they return, the symptoms are gone.

However, it's also possible that the condition that they saw the doctor for in the first place was an allergic reaction to another medicine. And if they stopped taking that initial medicine, well, this might be a case of an extraneous variable. Meaning what?

Well, meaning that it wasn't the medicine the doctor prescribed that caused the symptoms go away. It was related to something else. In this case, they simply stopped taking a medicine that was causing them to react to it in terms of having an allergic reaction.

Recall, if you will, from a previous lesson, the idea behind random assignment. Random assignment is done so we don't over- or under-represent a specific portion of the population. And we would randomly pick people in order-- or randomly pick observations in order to basically try to avoid having all the observations with the same characteristics.

So when conducting an experiment, there are two types of bias that require special attention. One is what's called "selection bias" and the other is called "participation bias." Selection bias can occur when the random assignment of individuals or things to the groups in an experiment is not done at all or not even possible.

A good example of selection bias would be a telephone poll where a poller or an experimenter decides to call people and ask them questions about a particular issue, and they pick the landline numbers based upon public record. So if they look in a telephone book or a public directory and they look at landline numbers, what they are doing here is they are specifically under-representing all people who might have a telephone. If they were only looking at landlines, they're going to miss people that have a mobile phone. It's very possible that someone might have a mobile phone but not a landline, and they would not be contacted in a situation like this.

And so selection bias would be where we're only choosing people that have a particular type of a telephone number-- a landline-- and we're under-representing the population. And so it's going to introduce bias into the results here. Observations that are left out of the study like this may cause a study to not represent the larger group of individuals being studied.

Participation bias is a special type of selection bias and can occur when participants have a choice to participate or not participate in an experiment. Those who participate or choose not to participate may have very different characteristics that can influence the results of an experiment.

So you may have been offered a chance to take a survey, and the survey gives you the opportunity to win a prize. And so, say, you happen to be a retailer and they hand you the receipt and circle the code at the bottom of it and say, well, if you type in this website and answer the questions, you're entered to win a $5,000 gift card. What would prompt somebody to participate in that knowing that they have a possibility of winning a gift card? In a case like that, it's what's called "participation bias."

Now, you may figure it's not worth the extra few minutes it's going to take, or you may decide, "You know, I'm going to do this," and then you lose the receipt or forget all about it. The idea here is that there is a difference between selection bias and participation bias. The idea behind selection bias is we're simply not choosing people randomly. We're not choosing observations randomly. However, with participation bias, it's entirely up to the individual whether or not they choose to act or not.

Selection bias is associated more as the result of an error by the researcher in selecting participants whereas participation bias is the result of the choices made by those participants. Since there are a lot of possible sources of bias in an experiment, it is extremely important for a researcher to be aware of possible sources of bias.

Now, remember, there are multiple steps in the experimental methods process. In Steps 2, 6, and 7 of the process-- whether we are coming up with a hypothesis, drawing a conclusion about an experiment, or simply trying to revise our hypothesis-- taking into consideration bias and its effect of the experiment leads to stronger conclusions.

With an experiment, there may be multiple variables that can impact the response variable being studied. Typically, errors in an experiment can be the result of variables that are not considered by the researcher. Variables like these are called "extraneous variables."

When selection or participation bias occurs, the groups being studied might have features that are impacted by an extraneous variable that was not considered. If using a random assignment, the results of extraneous variables is reduced. This is due to not assigning many participants with the same characteristics to a specific group.

When there is no random assignment and a lot of participants have the same characteristics, the extraneous variable may impact the group being tested in the same way-- and therefore, alter the results of the data. And so in the example of the SAT prep course, the extraneous variable is simply how much time the students spend studying outside of class. It is very possible the experimenter could draw a false conclusion based upon the fact that they simply took the class to begin with and did not consider how much time they spend studying outside of the class.

When reading statistical results, it is important to look for such extraneous variables since they can be responsible for misleading or even incorrect results. And therefore, one needs to take them into consideration when working in the experimental methods process, much like one takes into consideration bias.

So let's take a moment and go over the objectives and make sure we covered every one of them. The first one was to understand what bias is and how it can affect the outcome of an experiment. And so we defined "bias," and we talked about two key types of bias-- selection bias and participation bias.

The second was to know what an extraneous variable is and how they can also impact an experiment. An extraneous variable is simply one that is not being considered in the experiment, and that can have a significant effect on the outcome of the experiment.

So again, my name is Dan Laub. And hopefully, you got some value from this lesson.

Terms to Know

An error caused by the experimental design creating a situation where the values related to parameters of a population will consistently be over or under estimated.

Extraneous Variable

Variables that are not controlled in the experimental design.