Nonresponse Bias

This tutorial is going will explain nonresponse bias by examining:

1. Nonresponse
2. Nonreponse Bias

1. NONRESPONSE

A nice way to think of sampling is to use a "pot of soup" analogy. You want a representative sample, right? Well, you don't need to drink the entire pot of soup in order to figure out what's in it. You just need the right taste.

It would be like selecting all of the ingredients from the soup in a single tasting, but certain things can go wrong with the taste test that can affect what you think is in the soup. Just like you don't really know what the population looks like, you really don’t have a clear idea of all the ingredients in the soup. All you get is the taste, and if you don't get the right taste, you're going to leave something out and not know exactly what's in the soup (or, population).

It terms of sampling, nonresponse means that someone selected for the sample either can't be contacted or is unwilling to participate.

• NONRESPONSE
• Nonresponse is a lack of response from people you've selected. It affects the ability to draw conclusions from your sample.

Nonresponse happens. It's an inevitability that you will get uncooperative people, people that don't want to take your survey, or people who refuse to be part of your experiment. It may be just that you just won't be able to contact certain people.

2. NONRESPONSE BIAS

The problem of nonresponse is not a problem until the people that weren't able to be contacted or refused to participate-- differ substantially from the people that were in the sample. Now the sample is not representative of the population. That's called nonresponse bias because you're not getting an accurate cross-section of opinions. The opinions of people that you wanted to get are left out.

• Nonresponse Bias
• Bias that occurs when the people who were unable to be reached or unwilling to participate in a sample have substantially different opinions than the people who were included in the sample, resulting in a misrepresentation of the population.

IN CONTEXT

A workplace wishes to survey 200 of its 1,000 employees about their workload and their stress level, so they put 200 surveys in the workers' mailboxes. It’s likely that the people who have the biggest workloads might get left out of the sample because they don't check their mailboxes as often as other people. Or if they do get around to checking their mailbox, they may not complete the survey, or don't return it, because they're so busy.

What effect might that have? The 200 respondents that completed the survey may have reported that workload level is not that high. The only problem is that the people with the lower workloads are the only people who turned them in, because they had the time to take it.

Also, the people with the higher workloads didn't have the time to take it. The company might think the workload level is lower than it really is.

Take a look at these different ways of conducting a survey, or a poll, or a sample. Which of these methods, mail, telephone, or face-to-face, do you think has the highest nonresponse rate?

The answer is the mail. People will either throw it away, forget to fill it out, or maybe they'll fill it out and then forget to mail it back. This is problematic because when the United States takes its census of everyone in the country, it does it by mail. Sometimes they have to do follow-ups.

The nonresponse rate is easy to calculate. You just subtract the number that you got back from the number that you mailed out, and that's your nonresponse rate. Say you mailed out 100, and you only got 80 back. Well, that's 20 out of 100, or 20% nonresponse rate. In samples with high rates of nonresponse, follow-ups typically are needed.

So, supposing you started with a mailing, you might need to follow up by calling them at home. And if you can't reach them by calling them at home, you might need to follow up by coming directly to their house. And sometimes, even then, even when they are contacted, someone will refuse to participate.

Follow-ups like this might be more necessary in some areas of the country than others because different areas of the country have different rates of nonresponse.

On the other end of the spectrum is when people are excessively passionate about a topic and they’re eager to participate. The people who raise their hand to participate are volunteering their time because they have a strong opinion about the topic at hand. Participation bias happens when people participate because they have strong opinions about the topic, or they’re ambivalent because they are only participating because they are getting paid to participate.

IN CONTEXT

You need to gather information on an upcoming election and you ask people to participate in a focus group. In your group, you find that you have a group in strong support of the Democratic party and you have a group in strong support of the Republican party, and no one in the middle.

To correct this, you decide you’re going to pay participants $20 for their time. Now your group is filled with people who will simply tell you want they think you want to hear, which invites participation bias.

• Participation Bias
• Bias that occurs when participation in a study is voluntary. People who feel strongly may be the only participants.

Nonresponse bias occurs when people who are selected for the sample can't participate, either because you can't find them, or because they're actively refusing. The biggest problem is that if you have high rates of nonresponse, it might give you an inaccurate representation of what's going on with your population. You won't be able to use your sample to draw an inference about your population.

Good luck!