Or

4
Tutorials that teach
Significance Level and Power of a Hypothesis Test

Take your pick:

Tutorial

Hi. This tutorial covers significance level, also known as the probability of making a type I error. All right. Let's start with an example. Suppose you're interested in showing that the majority of people in your neighborhood support a proposal for a new city ordinance. If you're going to address this with a hypothesis test, we would need to start by writing a pair of hypotheses.

Let's start with the null hypothesis. Remember, the null hypothesis, we need to start with our population parameter. And the null hypothesis is always an equality statement. If we're thinking about a majority, we're thinking about a proportion.

And our null hypothesis is, we're going to assume that the population proportion equals 50%, or 0.5. What we're going to do is just let our null hypothesis be that p is equal to 0.5, where p represents the proportion of the neighborhood population that support the new ordinance. And then we would also need an alternate hypothesis. If we're trying to show that the majority of people support, our alternate hypothesis is that p needs to be greater than 0.5.

If we are then going to run our hypothesis test, we would eventually come up with some sort of conclusion. And what we want to do is, you plan on citing the results of the test in a letter to the editor in a local newspaper. What we need to think about are the two different errors that can pop up in this type of hypothesis test. Let's consider the type I and type II errors in this setting.

A type I error is, you conclude that a majority of residents support the proposal when no majority is actually present. And a type II error is, you conclude that the majority of residents do not support the proposal when a majority is actually present. We want to kind of think about which type of error may be-- the implications, I guess, of each type of error.

If we look at a type I error, in this case, if you're going to-- if this is going to be the results of the test and we end up putting this in the newspaper, a type I error here would basically be false advertising. You would be using a false conclusion to support your claim.

Now, a type II error, chances are, if you concluded that a majority of residents do not support the proposal, you probably wouldn't put that information in the newspaper in general. I would say, in this situation, a type I error might be a little bit worse.

Depending on the context of the problem, and what would be the impact of a type I error, is where we set a significant level. Significance level is the probability of making a type I error. In that last example, we would want the probability of making that type I error to be pretty small. And the significance is abbreviated with the Greek letter alpha. I generally make alpha like that.

Prior to running a hypothesis test, a level of significance must be decided upon. We have some commonly used significance levels. We commonly use the 1% significance level, a 5% significance level, and a 10% significance level. So the alpha values would be 0.01, 0.05, or 0.1. And the 5% level is a common default significance level. You'll see, most hypothesis tests are run at a 5% significance level.

In this case, if you're doing the newspaper example, you would want your alpha value, probably, to be pretty low. So I'd say 0.01 or 0.05, probably would be OK running your tests at either of those levels.

Now, the one thing is, is that if you decrease your level of alpha, your likelihood of making a type II error goes up. Decreasing alpha increases the probability or the likelihood of making a type II error. All right. This has been your tutorial on the significance level, also known as the probability of making a type I error. Thanks for watching.

Hi. This tutorial covers the power of a hypothesis test. So let's consider the hypotheses for a possible test. I've simplified these hypotheses. So we would be testing for a parameter, but I've simplified them so that the null hypothesis would indicate that no cancer is present. And the alternative hypothesis would indicate that cancer is present.

So if we think about the implications of each type of error, a type I error would be that the test indicates cancer is present when the patient does not actually have cancer. And a type II error is the test indicates cancer is not present when the patient does actually have cancer. So obviously, these are both errors and neither of them-- we wouldn't want either of these two things to happen.

But I would say with a type I error, if you're indicating cancer is present when the patient does not actually have cancer, so in this case, the patient would probably be subjected to more tests, possibly some treatment like chemotherapy or something like that. So it would waste a lot of resources. It would also put unneeded stress on the patient.

But I would say in this case, a type II error would be worse. So if the test indicates that cancer is not present when they actually have cancer, obviously they're not going to be subjected to timely treatment they need. So I would say a type II error in this case would be a pretty bad mistake to make.

So let's define the power of a hypothesis test. It's the probability of not making a Type II error. So the power is a good-- something that's good. So we want the power of the hypothesis test to be pretty high. We're not going to calculate the power of the test here. But the power of the test is almost always calculated with technology. And like I said, we want the power to be high.

So there are two simple ways to increase the power of a test. So if we increase the level of significance, so that value of alpha, we're going to be able to reduce the-- or increase the power of the test.

Now, one thing, though, that can happen is that if you increase the level of significance, you're going to increase the risk of a type I error. So if we lower the probability of making a type II error, increasing the power, we're going to increase our risk of making a type I error.

And the second way is that you're going to increase the power of the test is just to use a larger sample. And a lot of times, this isn't necessarily going to be an option because of cost constrictions or time constraints, things like that. But know that using a larger sample will give you a higher power. This has been the tutorial on the power of a hypothesis test. Thanks for watching.