Response Bias

This tutorial is going to discuss response bias by focusing specifically on:

1. Response Bias

1. RESPONSE BIAS

Response bias is when people's answers are influenced. A nice way to think about sampling is with a pot of soup analogy. When you get a representative sample, that's like getting a little taste of everything in the soup.

But things can go wrong and you don't get the right taste of the soup. Response bias can occur if the wording of the question is unclear to the respondent, or if a respondent is uncomfortable due to sensitive or personal nature of the questions, or if the respondent feels like the questioner is implying that the question has a quote unquote "correct" response. That's also called social desirability bias.

• Response Bias
• Bias that occurs when either (1) the question is poorly worded so that certain responses are over-represented, or (2) the respondent is confused by the question or feel like they should lie due to the sensitive nature of the question.

IN CONTEXT

On April 20, 1993, the New York Times published an article on a survey conducted by the Roper Organization on behalf of the Jewish American Community about the soon-to-be opened Holocaust Museum in Washington, DC, right here.

The newspaper reported that 22%, an astounding number of adults surveyed, expressed some doubt as to whether the Holocaust had actually occurred. The actual question that was presented to people was:

"Does it seem possible, or does it seem impossible to you, that the Nazi extermination of the Jews never happened?"

This seems to be a fairly straightforward question, but there was a big problem with it, and it caused response bias. The problem is that the question contained a double negative.



The good thing is that, one year later, the question was revised, and it became clearer. The new question stated:

"Does it seem possible to you that the Nazi extermination of the Jews never happened, or do you feel certain that it happened?"

With this new, clearer question, the question clearly distinguishes between what the two options are-- "does it seem possible," or "do you feel certain?"

With the two options clearly defined, less than 2% of individuals were unsure as to whether it was real or not. This gives a more accurate interpretation of what the American public feels. So, unclear questions can lead to an inaccurate representation due to response bias.

The other scenario in which this can occur is when people will answer a question because they are either ashamed, or they think that there's a right answer that someone's fishing for.

   A survey about drug use may exhibit many people saying they've never used it, even if they have. And even if there's no consequence, and even if the survey is anonymous, they'll still say they've never used when, in fact, they have.

There are certain topics that are particularly sensitive and might make a person want to lie:

• Criminal history - they might say they don't have one, even if they do.
• Sexual behavior - highly sensitive and personal nature
• Racial prejudice - there's an implied right answer; people don't want to say that they're racially prejudiced.
• Income-- people will report it as being higher than it actually is if they're of low income status, or even possibly more surprisingly, people will report it as lower than it really is if they're of a very high income status. A lot of people don't want to be very showy about their wealth, and so they'll try and come up with a more-- in their eyes-- reasonable number.

How does this affect what we think about the population? How does this affect the "soup?" I's like taking a sample of the soup and only tasting the things that you want to taste. Maybe you don't like beans, and so you just sort of ignore the fact that they're there. You don't get the overall flavor of what's supposed to happen.

Same thing with response bias. It doesn't give you the right overall interpretation of what this is supposed to be like.

Response bias occurs one of two ways. Either a respondent doesn't understand the question and so gives an answer that he wasn't intending, or because the respondent wants to give a supposedly correct answer to the questioner.

Both of these can be inaccurate representations about what actually is the truth about the population. Response bias is a tough thing to get rid of, especially when it deals with the wording of the questions and it's unintentional.

Good luck!