Free Professional Development

Or

2
Tutorials that teach
Determining Significance Based on Probability

Take your pick:

Tutorial

Source: All graphs created by Dan Laub; Image of runner, PD, http://www.clker.com/clipart-woman-runner.html; Image of rate monitor, PD, http://www.clker.com/clipart-26388.html; Image of dog, PD, http://www.clker.com/clipart-11682.html; Image of iPhone, PD, http://www.clker.com/clipart-iphone-g.html; Image of beach chair, PD, http://www.clker.com/clipart-beach-chair-and-umbrella.html; Image of gas pump, PD, http://www.clker.com/clipart-blue-fuel-pump.html

Hi. Dan Laub here. And in this lesson, we're going to discuss determining significance based upon probability. But before we get started, let's cover the objective for this lesson. By the end of this lesson, you should be able to understand the relationship between the level of significance and the probability that an event occurs. So let's get started.

Remember that in the experimental method, when determining whether or not two variables have a cause and effect relationship, the null hypothesis states that there is not a cause and effect relationship between the two variables, whereas the alternative hypothesis states that there might be a relationship between the two variables. The results of an experiment are considered significant if they are unlikely to occur when the null hypothesis is true.

Probability is a means of establishing how likely an event or a result of an experiment is. A p value is the probability that a result occurs in the event that the null hypothesis is actually true. Results of an experiment are considered significant if they are unlikely to occur in the event that the null hypothesis is true. Similarly, one could say that the results of an experiment lie in the region of rejection with area of 5%, and such a result is significant when its p value is less than the significance level of 5%.

For example, suppose that we were to consider the relationship that exists between how often a person engages in cardiovascular exercise and their resting heart rate. The null hypothesis for such an experiment would be that regardless of how often one exercises, it would have no impact on their resting heart rate. However, since there is a cause and effect relationship between the frequency of cardio exercise and resting heart rate, we would expect the results of such an experiment to be significant.

Let's suppose, in this case, the p value turns out to be 0.02, or 2%. As you can see on the graph, the combination of the two-tailed region works out to be 5% of the entire region under the normal distribution curve. The results of this specific test resulted in a p value located at this point here, which lies on the region of rejection since it is less than the 5% level of significance.

As another example, let's look at the relationship that exists between how many text messages a person sends per day and the number of pets that they have. Let's also suppose that we have a pre-established level of significance of 5%. In a case like this, we would expect to fail to reject the null hypothesis, and as such, would not be surprised to arrive at a p value which is 35%. Such a value would indicate that the results of the experiment are not significant, because it is greater than 5%.

What about the case of the average price of gasoline and the average annual number of miles traveled on vacation by Americans? Suppose that having conducted two experiments on these variables, that we arrived at results that yielded p values of 1.5%. This is clearly expected given the nature of the relationship that exists between the two variables, and as such, means that the results are significant.

So going back to the objective, let's make sure we covered what we said we would. We wanted to be able to understand the relationship between the level of significance and the probability that an event occurs, which we did. We went through several different examples illustrating how that p value tells us whether or not there is a good probability that an event would actually take place or not.

So again, my name is Dan Laub. And hopefully, you got some value from this lesson.