Survey Language

Video Transcription

This tutorial is going to talk to you about survey language. And the thing about surveys is you have to be very precise about what exactly it is that you want to measure. Because otherwise, you might steer someone towards answers they might not normally get. So the questions must be worded very, very carefully so those conducting it can get the accurate responses that they're looking for. I'm going to use the always controversial topic of taxation.

So suppose that these two survey questions appeared on a survey. But one is one type of wording and one is another type of wording. They're asking the same question. If you take a look, all this does is explain in a little bit more detail what the estate tax is. When a person dies, his or her heirs pay taxes on the amount of his or her estate that exceeds $1 million. This is known as the estate tax. What is your opinion regarding the fairness of such a tax? Versus let's look at this one. When a person dies, a portion of his or her estate is subject to an estate tax. Would you say such a tax is-- and the same answer choices.

Now which one of these do you think has more people saying the tax is fair? If you said the top one has more people saying the tax is fair, you would be correct. This inclusion of $1 million makes people think oh, the tax doesn't really apply to me. I don't have a million dollars in assets. Sure, it's fair to tax the people whose assets are more than $1 million, versus this one that simply says that a portion of an estate is subject to tax.

Now, I'm not saying that any one of them is more fair than the other. I'm saying that either wording of the question can be used to push a particular agenda. Now, those are both better than this example here, which is particularly inflammatory. Do you favor repeal of the death tax, so that many families won't be unfairly burdened with hefty taxes at the time of their grief? Notice it's appealing to their emotion. Because they're saying that the family is so grieved. It says the word tax twice. It calls it hefty. And it calls it by its slang term, the death tax, which sounds just terrible.

Now both examples on the previous page are much more appropriate than this one. This one contains very loaded language. And we want to stay away from that if we're going to get useful responses. Other kinds of questions might not be so inflammatory. They might just be very confusing to the reader. And you might not get reliable response from it. So for instance, to what extent did you use the course website and video tutorials provided by your teacher to help you learn the material for the course? Well, the problem is this one actually is asking two questions at once. It's asking to what extent did you use the course website, and to what extent did you use the video tutorials?

Now maybe someone used the course website without watching the tutorials. They couldn't really answer this question accurately. This one's called a double barreled question. Now how about this one. Do you agree or disagree with the following statement? Teacher should not be forced to create a course website for every course they teach.

Again, this one's a little confusing because of the disagree and then not. This makes it a double negative statement. So by saying you disagree, you're saying that you don't think teachers shouldn't create course websites. So you're saying that you do think that they should have to create course websites. It's very confusing. And the wording makes the logic much harder to follow than it really should be. This question can be made a lot more simple by removing the word not.

So there's all sorts of different ways to cloud the intent of a question by using appropriate language. The language can be loaded, or it can be double barreled, or it can just be confusing with a double negative. So to recap, if the wording of the questions in a survey favor certain outcomes, or it's unclear, you can't use that survey to gain any useful information, which is why you have to be very, very careful at the beginning of your survey to make sure that not only are you understanding what you're trying to obtain from the survey, but how you're going to ask it in order to get reliable answers. Good luck. And we'll see you next time.