Selection and Deliberate Bias

1. Selection Bias

You may recall that sampling is like a pot of soup. Selecting a little bit of each ingredient for the soup is like obtaining a representative sample for an experiment. However, things can go wrong with the taste test, which may limit the ability to draw conclusions about the pot of soup as a whole.

Selection bias is also called undercoverage bias. It occurs when a significant subset of the population is left out of the sample. This is not necessarily intentional, but rather, occurs when they were systematically ignored by whoever was taking the sample.

IN CONTEXT

In 2008, almost every poll showed Barack Obama leading by at least five percentage points leading up to the New Hampshire presidential primary. All of these were based on random digit dialers calling a random sample of New Hampshire households. It was a well-done survey of all accounts.



However, what happened was that Clinton gained some support in the last few days. Mainly, a lot of college students ended up coming out in support of Hillary Clinton in the last days when people were expecting all college students to come out in support of Obama.

Because a lot of the college students are from out of state, they aren't actually New Hampshire residents. For that reason, they were not counted and, as a result, the sample got every prediction wrong and Clinton ended up winning.

term to know

Selection Bias

A bias that results from systematically excluding certain subsets of the population from the sample. It is not necessarily intentional.

2. Random Digit Dialing

The New Hampshire primary used random digit dialers. Random digit dialing involves using a machine to select random phone numbers from within selected area codes. It doesn't randomly select the area code necessarily, but once it's in the area code, it can randomly select digits and dial that particular phone number after which the poll can be conducted.

The biggest advantage of using random digit dialers is that they can reach mobile phones and unlisted numbers that you wouldn't be able to obtain using a phone book. So, it evens the playing field a bit since anyone can be selected for that sample as long as the phone number is within that particular area code.

think about it

How does selection bias affect what we think is in the soup? Imagine that certain ingredients were located only in certain locations in the pot. Maybe noodles sunk to the bottom. If you tasted only from the top, it doesn't matter how big that taste is. If you missed the noodles, you wouldn't even know they were there. That's the same as dealing with selection bias. Because you didn't select the representative group of ingredients from the population, you don't get the right idea of what's going on. It limits your ability to generalize your findings to the general population.

term to know

Random Digit Dialing

A method of contacting people on the phone. Random numbers are dialed, so this allows researchers to sample people with unlisted phone numbers.

3. Deliberate Bias

Deliberate bias is exactly what it sounds like: it's a bias that's done on purpose. While deliberate bias doesn’t happen very often, it can occur when there's a conflict of interest between the people performing research and the people funding--who are usually the ones benefiting from--that research.

Typically deliberate bias is motivated by an interest unrelated to the integrity of whatever you’re researching. Most research is done with integrity, but when personal prestige, the advancement of some ideology, or money get in the way, it’s harder to prove that intentions are pure. Politics can be an industry ripe for deliberate bias. Perhaps people call with a poll, but the survey includes a leading question to cause the person to respond in a certain way. When this is done it's called “push polling” and it’s highly suspect.

IN CONTEXT

Deliberate bias can happen in other areas too--even the medical field. Suppose there are two drugs: Drug A and Drug B. The company for Drug B posed the following leading question:

“If Drug A was linked to cancer, would you be:

• more likely to choose Drug B?
• less likely to choose Drug B?
• equally likely to choose Drug B?”

Based on how this question was posed, Drug B would be more likely to be chosen.

But there’s more. They've put a thought into the participant’s head that Drug A is linked to cancer. Did they ever explicitly say that? No, they said if it was linked to cancer. However, now they've placed the association in the participant's mind. Subconsciously they're beginning to steer consumers away from Drug A and towards Drug B.

If a drug company funds a study to determine if it's latest drug is effective, the researchers stand to gain a lot of money and prestige for having tested the drug, if proven effective. For this reason, they might not be the best choice to test the drug.

IN CONTEXT

An environmental research group is hired by a real estate developer to investigate the effects of a new building. If the results are favorable, they might get another contract with that real estate developer. If the environmental research group doesn’t come through with a favorable interpretation, another group will, and that group will get the next contract.

The environmental research group wants to be hired by the developer on another project, so there is a conflict of interest.

term to know

Deliberate Bias

The purposeful misrepresentation of data for the purpose of advancing an agenda.

4. Unintentional Bias

Unintentional bias occurs when there is simply an error in the design of the study. Two types of unintentional bias include:

• Response bias, which involves the wording of questions or refers to people feeling like they have to lie.
• Selection bias, which involves how the sample was selected, such as when people are not included in the selection process, even though they make up a portion of the population.

term to know

Unintentional Bias

Bias that is not purposeful. It exists because of errors in the design of the study.