Using the Wrong Sampling Frame

Video Transcription

This tutorial is going to teach you about the problem of using the wrong sampling frame. This is an error that we can make when we sample.

Now a nice way to think about sampling is with an analogy like a pot of soup. And we want to select a representative sample, which would be like dipping your ladle in and getting a little bit of every ingredient from the soup. But when we taste test we can get the wrong idea if what's in the soup. And we can affect our judgment.

So one problem might be choosing the wrong sampling frame. Now a sampling frame is the entire set from which we're going to sample. Now you might think to yourself, well isn't that the entire population? Don't want to generalize our findings to the population? And, yes, ideally that would be the case. But we don't always have the entire list of population at our fingertips. If we were doing a sample for like a school then we would have the entire population at our fingertips. Because all we'd have to do is check out the attendance. But we don't always have that luxury.

Suppose that we were doing a sample of a city. And we choose to do a telephone survey. The sampling frame, in that case, is all the individuals in the population of the city that have telephones. Now you might wonder is that a big deal. Most everyone has a telephone. And I would agree with that. The only difference is the people without phones cannot be included in the sample at all, even if they wanted to be. And using this sampling frame, as opposed to the entire population, is only problematic if we feel like the excluded subset, that is the people without phones, feel substantially different about the topic that we're going to be asking them about than the people within the sampling frame.

So for instance maybe we were doing a phone survey about telephones. It might be that people without telephones feel differently about phones than people who have them, on a count of they don't have them. So the only way that this could result in bias is if we thought that there was a huge difference between the people that are part of the sampling frame and the people that aren't part of the sampling frame.

Now the question is how does this affect the soup? How does this affect our ability to draw inferences about the population? Well the analogy here is this would be like setting the soup on the stove and cooking it up but leaving out some ingredients. So some ingredients that were called for in the recipe were left out. You might get an accurate representation of what is in there. But what is in there is not the entirety of what should have been in there. And so you won't get the accurate reflection of what the soup should have tasted like had you included everything.

So to recap it's important to choose the correct sampling frame and to use it if you can. If the entire population is available to be selected for the sample use the population as your sampling frame. And sample from there. If it's not available to be selected then the conclusions that you can draw might be a little bit limited. And this might be the limits of using the wrong sampling frame. Good luck. And we'll see you next time.