Source: Image of P-value graph p-value Graph, Creative Commons: p-value graph: http://en.wikipedia.org/wiki/File:P-value_Graph.png
This tutorial talks about test statistics. A test statistic tells us how far a sample statistic is from the assumed parameter if the null hypothesis is true. And then it does this in standardized units.
The way we calculate the test statistic is by taking our statistic subtracting the parameter and then dividing by the standard deviation of the statistic. Different tests calculate the standard deviation of the statistic in different ways, but the overall formula of statistic minus parameter divided by the standard deviation of the statistic stays the same.
Now if we're talking about z-scores, if you obtain a z-score that is far from 0 in the direction specified by the alternative hypothesis, then you found evidence against the null hypothesis. We can also look at p-values. A p-value is the probability that the test statistic is that value or more extreme in the direction of the alternative hypothesis.
So here if we're looking at our distribution, in the middle, we have the range of the most likely observations. And then at the tails, we have the very unlikely observations and very likely observations. If we plot our observed data point, the p-value-- that shaded green area-- is the probability that that observed value or more extreme than that comes up by chance.
One thing to note in this picture, it's showing a right-tailed test. So here, it's only showing what a right-tailed test would look like. There's also left-field tests and two-tailed tests.
Now we can also talk about critical values. A critical value is a value used when deciding whether to reject or fail to reject the null hypothesis, if the test statistic is beyond the critical value, then we reject the null hypothesis.
Now the critical value that we choose corresponds to a particular significance level, and that significance level is different for each type of hypothesis test. With a two-tailed test, we have two symmetric critical values. There'd be one for the upper tail and one for the lower tail.
So this has been your tutorial on test statistics.