Survey Language

This tutorial will discuss how the language used in a survey can shape the results that survey obtains. You will cover:

1. Survey Language
2. Loaded Language
3. Confusing Language

1. SURVEY LANGUAGE

This tutorial is going to talk to you about survey language. And the thing about surveys is you have to be very precise about what exactly it is that you want to measure. Because otherwise, you might steer someone towards answers they might not normally get. So wording of questions must be worded very, very carefully so that those conducting the survey can get the accurate responses that they're looking for.

Wording of Questions

The way in which a question is phrased can influence the response given by the participants in the study.

IN CONTEXT

Suppose that these two survey questions appeared on a survey. The first uses one type of wording and second uses another type of wording, but they're asking the same question.

If you take a look, all the first one does is explain in a little bit more detail what the estate tax is: “When a person dies, his or her heirs pay taxes on the amount of his or her estate that exceeds $1 million. This is known as the estate tax. What is your opinion regarding the fairness of such a tax?” So let's look at the second one: “When a person dies, a portion of his or her estate is subject to an estate tax.” The answer choices on the second one are the same.

Now which one of these do you think has more people saying the tax is fair? If you said the top one has more people saying the tax is fair, you would be correct. By including information about the level of income at which the estate tax applies (which is $1 million), this survey leads people to think that the tax doesn't really apply to them since it is unlikely that the average person answering this survey has a million dollars in assets. So because the question makes the answerer feel removed from the population who might be taxed, the person answering the survey is more likely to think that it's fair to tax the people whose assets are more than $1 million. In contrast, the second one that simply says that a portion of an estate is subject to tax. I'm not saying that any one of them is more fair than the other. I'm saying that either wording of the question can be used to push a particular agenda.

2. LOADED LANGUAGE

Now, those are both better than this example here, which is particularly inflammatory.

This one contains very loaded language that we want to stay away from that if we're going to get useful responses. Notice how this question is appealing to the emotion of the person answering the question. It says the word tax twice. It says that the family being taxed is grieving. It calls the tax “hefty.” And it refers to the tax by its slang term “the death tax,” which sounds just terrible

In these ways, this wording is loaded so that the person answering the question is likely to be swayed one way. Therefore, this question will not elicit an unbiased response, but instead will get more people to answer that they favor repealing the tax.

3. CONFUSING LANGUAGE

Other kinds of questions might not be so inflammatory. They might just be very confusing to the reader, which means that you might not get reliable response from them.

What kind of response do you think this will elicit? Will the people you survey know how to answer this question clearly?

"To what extent did you use the course website and video tutorials provided by your teacher to help you learn the material for the course?"

Well, the problem here is that this question is actually is asking two questions at once:

1. To what extent did you use the course website?
2. To what extent did you use the video tutorials?

Now maybe someone used the course website without watching the tutorials. They couldn't really answer this question accurately. This one's called a double barreled question.

Again, what kind of response do you think this question will elicit?

Do you agree or disagree with the following statement? Teacher should not be forced to create a course website for every course they teach.

Again, this one's a little confusing because of the language; it uses both “agree or disagree” and “teachers should not.” This makes the question a double negative statement: by saying you disagree, you're saying that you don't think teachers shouldn't create course websites. So you're saying that you do think that they should have to create course websites. It's very confusing. The wording makes the logic of the question much harder to follow than it really should be.This question can be made a lot more simple by removing the word not.

So there's all sorts of different ways to cloud the intent of a question by using appropriate language. The language can be loaded, or it can be double barreled, or it can just be confusing with a double negative.

Loaded language in a survey creates bias. Confusing language is not inflammatory, but may ask 2 separate questions, which confuses the participant. Survey language must be precise in order to gather accurate information.

Thank you and good luck!