Or

4
Tutorials that teach
Statistical Significance

Take your pick:

Tutorial

This is your tutorial on statistical significance. Statistical significance is a quantitative assessment of whether or not observations represent a pattern or they're just due to chance. So when we talk in other tutorials about something being significant, we're talking about statistical significance.

There's also something called practical significance. Practical significance is whether or not something is meaningful in the real world. In practice, does this affect our lives? Something that's important to remember is that with a large sample size, we can get to find some things that are statistically significant that might not be practically significant. And there are other things that might be practically significant but not statistically significant.

Statistical significance and hypothesis testing are connected. With hypothesis testing, we are determining what level of difference from an assumed claim is going to constitute statistical significance. That's an important part of it. And then, on the end of it, a result is called statistically significant if it is unlikely to have occurred by chance.

So this first example says a state survey of all high school students finds that 15% of 10th graders drink regularly. A town randomly selects 100 students and finds that 18% of their 10th graders drink regularly. So by doing some sort of statistical test and setting a significance level, and if this passes that test, essentially, then we can say that, yes, this is statistically significant, or no, it's not.

Now, whether or not it's of practical significance is, does this affect our life in the real world? And for this town, even if it came back that, no, there was no statistical significance, that it's random that 18% of our 10th graders drink regularly, you still might want to do something about that. It still might have meaning for you in the real world, that difference, because it's about something that's serious.

So this here may or may not have statistical significance. We don't know. We'd have to do a test. But it might have practical meaning for us. We might want to do something about it. Here, example 2 says a statistical study finds that people who talk on their cell phones have a statistically significant greater risk of developing cancer, 1 in 20 million instead of 1 in 10 million, which is for non-cell phone users.

So in this case, using a cell phone doubles your risk of getting cancer. And it says that there is statistical significance. However, is this of practical significance? Going from a 1 in 20 million to 1 in 10 million chance of cancer for not using a cell phone might not mean much in the real world. People are still probably going to use their cell phones because they're very attached to them and there's a lot of advantages for what is a very, very small decrease in risk. So here's a case where even though it's statistically significant, there might not be much real-world application of it and much practical significance. This has been your tutorial on statistical significance.