Online College Courses for Credit

+
Collecting Data: Quantitative Approaches

Collecting Data: Quantitative Approaches

Rating:
(0)
Author: Sophia Tutorial
Description:

Identify the methods used to conduct quantitative research in sociology.

(more)
See More
Tutorial
what's covered
In this lesson you will learn about the guiding principles around using research methods that result in numerically measurable data. You will explore specific examples of how these methods are used and what their strengths and weaknesses are. You will continue to use this knowledge to build on your problem solving skill. Specifically, this lesson will cover:
  1. Principal Research Methodologies
  2. Experiments
    1. Lab Settings
    2. Natural or Field-Based Experiments
  3. Survey Research
    1. Questionnaires
    2. Interviews
  4. Secondary Sources
  5. Combining Methodologies

1. Principal Research Methodologies

Return to step four of the scientific method: design a research plan. You design a research plan according to a research methodology, which is a systematic and coherent plan for conducting research. Research is vital to social science and is most easily used when it is quantitative, or countable. Data expressed in numbers shows validity and reliability in the method and in the outcome. Most importantly, things that can be counted can also be compared and contrasted, analyzed and tabulated.

In planning the design of a study, sociologists generally choose from four widely used methods of social investigation:

  • Experiments
  • Survey research
  • Secondary data analysis
  • Field research
The topic will shape the research design, while research design likewise shapes the study topic. Researchers choose methods that best suit their study topics and that fit with their overall approaches to research. Every research method comes with advantages and disadvantages, and the topic of study strongly influences which method or methods are put to use. In this section, you’ll examine how researchers use each of these research methods.

EXAMPLE

Sociological research is used by companies to evaluate processes in the distribution of goods. This kind of research can be quite complex, requiring research into a company's current and past rates of productivity to find out how efficiently the organization is fulfilling its orders. But it will also involve participant observation on the warehouse floor, as well as interviews with managers and staff. In this way the researchers will take into account both qualitative and quantitative research to inform their outcomes.

term to know
Research Methodology
A coherent and organized plan for conducting research.

2. Experiments

An experiment is a regimented, highly controlled research method for investigating cause and effect relationships between variables.

You’ve probably tested some of your own theories: “If I study at night and review in the morning, I’ll improve my retention skills.” Or, “If I stop eating junk food, I’ll feel better.” Cause and effect. If this, then that. Causation is the existence of a cause and effect relationship between variables, and it is difficult to establish, so even if we seem to find evidence in our own lives that appears to prove our hypotheses, this is not sociological research nor is it evidence of causation. Sociologists set up specific studies within prescribed parameters in order to examine relationships between variables.

big idea
The primary difference between our everyday observations and sociological research is the systematic approach researchers use to collect data.

Experiments aim to measure the relationship of the independent variable to the dependent variable, and the researcher or research team will attempt to control all other variables in the experimental process. This is often done in a lab-based setting, but can also be done as a field experiment. Some experiments look for correlation', or how two variables change together. Others use controlled conditions to attempt to explain cause and effect. A controlled experiment is one where just one or a few factors are changed at a time, while everything else is kept constant. Researchers use controlled experiments to gain information about the impact of an independent variable by comparing the experiences or outcomes of participants who are exposed to the variable with participants who have not been exposed to the variable.

2a. Lab Experiments

To set up a lab-based experiment, sociologists create artificial situations that allow them to manipulate specific variables for their subjects.

In a traditional lab experiment, the sociologist selects a set of people with similar characteristics, such as age, class, race, or education. Those people are divided into two groups. One is the experimental group and the other is the control group. The experimental group is exposed to the independent variable(s) and the control group is not.

EXAMPLE

To test the benefits of tutoring, the sociologist might expose the experimental group of students to tutoring while the control group does not receive tutoring. Then both groups would be tested for differences in performance to see if tutoring had an effect on the experimental group of students. As you can imagine, in a case like this, the researcher would not want to jeopardize the accomplishments of either group of students, so the setting would be somewhat artificial -- a test that only counted for purposes of the experiment.

hint
It might be easier to understand the role of experimental and control groups in a lab experiment by thinking about a pharmaceutical drug trial, in which the experimental group is given the test drug and the control group is given a placebo or a sugar pill. This allows the researchers to see if the experimental group has better outcomes than the control group. The same principle applies for sociological research, except instead of just looking to see whether an intervention (the test drug for pharmaceutical research, the independent variable in sociological research) is effective, sociologists are looking to see what, if any, impact the intervention has at all.

2b. Natural Experiments

In a natural experiment, the experiment takes place in the subject’s natural environment. There are fewer controls but the information might be considered more accurate since it was collected without interference or intervention by the researcher. As a research method, either type of sociological experiment is useful for testing if-then statements: if a particular thing happens, then another particular thing will result.

In Context

In 1968, two researchers, Rosenthal and Jackson, wanted to see how teacher perception of student ability had an impact on their learning.

They took an elementary school and gave the students an assessment to measure IQ. After the assessment, the teachers were told that 20% of the students were more advanced than others and had greater potential because their IQ scores were higher. The students were identified to the teachers by name. However, the researchers were lying; the selected students were chosen at random, and did not actually have higher scores.

At the end of the experiment, the students were tested again. This time, the students who had been pointed out to the teachers as having a higher IQ actually did score higher than their peers on the test! The only thing that had changed was the teacher’s perception of the students, which had an impact on student outcome. Thus, the independent variable was the teacher's perception of certain students, and the dependent variable was the students' results on the second IQ test. The control group, who did not receive the independent variable, were the rest of the students.
Apply Your Skill
In this example, we see that teacher perception has an impact on learning. This can lead us to potentially solve problems with student achievement. Given the results of this study, researchers may then want to implement methods to improve teacher perception of student ability. Using the sociological methods outlined here, they can begin working to solve the identified problem.

Sociologists have long been interested in inequality and discrimination. Read the study below to see how one sociology professor sent her students to the field.

In Context

In 1971, Frances Heussenstamm, a sociology professor at California State University, Los Angeles, had a theory about police prejudice. To test her theory she conducted an experiment. She chose fifteen students from three ethnic backgrounds: Black, white, and Latino. She chose students who routinely drove to and from campus along Los Angeles freeway routes, and who’d had perfect driving records for longer than a year. Those were her control variables—students, good driving records, same commute route. These students all had safe, up-to-date cars and signed a pledge to drive safely.

Next, she placed a Black Panther bumper sticker on each car. That sticker, a representation of a social value, was the independent variable. Founded in Oakland, California in 1966, the Black Panthers were a revolutionary African-American group actively fighting racism. Heussenstamm asked the students to follow their normal driving patterns. She wanted to see whether seeming to support the Black Panthers would change how these drivers were treated by the police patrolling the highways (the dependent variable).

The first traffic citation, for an incorrect lane change, was made two hours after the experiment began. One participant was pulled over three times in three days. He quit the study. After seventeen days, the fifteen drivers with the Blank Panther stickers had collected a total of thirty-three traffic citations between them, regardless of the driver's race or gender, and the funding to pay traffic fines had run out. The experiment was halted (Heussenstamm 1971).

think about it
What kinds of ethical issues are present in Heussenstamm’s experiment? Were some students at greater risk than others? How do you think the experiences of each group (Black, white, and Latino) differed? Do you think gender would influence the interaction between student and police officer?

True experiments are rare in sociology for two reasons:

  • Ethics: It is not ethical to manipulate people in real life or to treat people differently based on one or more of their identities, without their informed consent. If the subjects have informed consent, it might affect the outcome of the the experiment.
  • No control over other mitigating variables: It is impossible to control all the possible variables in real life.
Most of the time, sociologists will use what are called natural experiments, or experiments that simply occur in the process of conducting field work. Due to some of the parameters for performing lab experiments on human beings and the ethical concerns, experiments are not a very widely used method in sociology

think about it
Imagine your sociology professor asked you to place a bumper sticker on your vehicle that said "peace on Earth" in Arabic, and asked you to sign an informed consent before participating in the study. Would you do it? Why or why not? How does geographic location and personal identity affect one’s experience and potential risk factors?

terms to know
Experiment
A regimented, highly controlled research method for investigating cause and effect relationships between variables.
Causation
The existence of a cause and effect relationship between variables.
Correlation
How two variables change together.
Laboratory Experiment
An experiment in which the independent variable can be manipulated between control and experimental groups to measure the outcome or dependent variable.
Natural Experiment
An experiment that occurs in the natural environment of the subjects.


3. Survey Research

Survey research is a research method in which subjects respond to the researchers’ questions directly, either in an interview or on a questionnaire.

As a research method, a survey collects data from subjects who respond to a series of questions about behaviors and opinions, often in the form of a questionnaire. But surveys can also take the form of interviews with open-ended questions and/or closed-ended questions. The survey is the most widely used scientific research method in sociology, and often multiple types of surveys can be used together in the same study.

EXAMPLE

A sociologist is doing research on business travelers who prefer to use hotels and those who prefer to rent apartments. The researcher conducts preliminary one-on-one interviews and generates some insights to test his hypotheses. He then calls a focus group together, one with the people who prefer hotels and one with the people who prefer apartments, to validate their responses in the interviews, bounce his ideas off of them and discuss his hypothesis to ensure that he is on track.

Surveys might seem innocuous. How could someone be harmed with a survey? However, as with all types of sociological research, sociologists must adhere to a list of ethical considerations and present their plans before an Institutional Review Board (or IRB) before they commence any type of sociological survey if it is to be used for research purposes.

EXAMPLE

If a professor asks students to complete a survey that asks about previous experience in an online class for the purpose of understanding students’ prior knowledge, that would not be considered research and would not raise any ethical concerns. If a faculty member wants to use the results of the research for an academic publication, it might require an Institutional Review Board approval as well as some additional precautions (such as a detailed informed consent, maintaining anonymity of subjects, etc.) since the faculty member is utilizing current students for research purposes.

Sociologists conduct surveys under controlled conditions for specific purposes. Surveys gather different types of information from people. Surveys can track political preferences, or patterns in reported individual behaviors (such as sleeping, driving, or texting habits), or can gather factual information on subjects like employment status, income, and education levels.

While surveys are not great at capturing the ways people really behave in social situations, they are an effective method for discovering how people feel and think—or at least how they say they feel and think. The standard survey format allows individuals a level of anonymity in which they can express personal ideas, but even with anonymity people won't always tell the truth, or the whole truth.

big idea
Surveys are the most widely used scientific research method in sociology, because the data can be easily organized and tabulated.

A survey targets a specific population, who are the people who are the focus of a study, such as college athletes, international students, or teenagers living with type 1 diabetes. Most researchers choose to survey a small sector of the population, or a sample: that is, a manageable number of subjects who represent a larger population. The success of a study depends on how well a population is represented by the sample.

In a random sample, every person in a population has the same chance of being chosen for the study. According to the laws of probability, random samples represent the population as a whole. For instance, a Gallup Poll, if conducted as a nationwide random sampling, should be able to provide an accurate estimate of public opinion using a relatively small sample. For polls focused on U.S. issues, a random sample of 1,000 is representative of the opinions of 230 million adults with a +/- 4 percentage points of accuracy. It’s amazing, isn’t it?

After selecting subjects, the researcher develops a specific plan to ask questions and record responses. It is important to inform subjects of the nature and purpose of the study up front. If they agree to participate, researchers thank subjects and offer them a chance to see the results of the study if they are interested. The researcher then presents the subjects with an instrument, which is a means of gathering the information. We will now look in detail at some of the most common survey instruments.

3a. Questionnaires

 A common instrument is a questionnaire, in which subjects answer a series of questions. For some topics, the researcher might ask yes-or-no or multiple-choice questions, allowing subjects to choose possible responses to each question. These '''quantitative data'''—research collected in numerical form that can be counted—are easy to tabulate. Just count up the number of “yes” and “no” responses or correct/incorrect answers, and chart them into percentages.

Questionnaires can also ask more complex questions with more complex answers. They can go beyond “yes” and “no,” or can offer a range of options next to a checkbox. In those cases, the answers are subjective and vary from person to person.

EXAMPLE

How do you plan to use your college education? What do you like about being a parent? How do you feel about climate change?

These open-ended types of questions require short essay responses, and participants willing to take the time to write those answers may convey personal information about religious beliefs, political views, and morals. Some topics that reflect internal thought are impossible to observe directly and are difficult to discuss honestly in a public forum. People are more likely to share honest answers if they can respond to questions anonymously.

This type of information is qualitative data— non-numerical data collected by the researcher through first-hand means such as: interviews, first-hand observation, questionnaires, focus groups, recordings, and original documents. Qualitative information is harder to organize and tabulate than quantitative information, but it can be much richer. The researcher will end up with a wide range of responses, some of which may be surprising. The benefit of written opinions, though, is the wealth of material that they provide. The advantage to using qualitative data is the in-depth knowledge a researcher can glean about the social lives of their subjects.

3b. Interviews

An interview is a one-on-one conversation between the researcher and the subject, and it is a way of conducting surveys on a topic. Interviews are similar to the short-answer questions on surveys in that the researcher asks subjects a series of questions. Participants are again free to respond as they wish, without being limited by predetermined choices, but in an interview they can be as expansive in their responses as they wish. In the back-and-forth conversation of an interview, a researcher can ask for clarification, spend more time on a subtopic, or ask additional questions. In an interview, a subject will ideally feel free to open up and answer questions that are often more complex. There are no right or wrong answers. But this type of qualitative data provides insight into institutions and communities that cannot be gleaned from multiple choice questions.

Questions such as “How did society’s view of alcohol consumption influence your decision whether or not to take your first sip of alcohol?” or “Did you feel that the divorce of your parents would put a social stigma on your family?” involve so many factors that the answers are difficult to categorize. A researcher needs to avoid steering or prompting the subject to respond in a specific way; otherwise, the results will prove to be unreliable. And, obviously, a sociological interview is not an interrogation. The researcher will benefit from gaining a subject’s trust, from empathizing or commiserating with a subject, and from listening without judgment.

There are multiple types of interviews, just as there are multiple types of questionnaires, including:

  • In-depth interview, which is usually one-on-one, unstructured, and lasts for a longer time, taking a more conversational tone
  • Focus groups, which are groups of individuals brought together in one room to engage in a guided discussion of a topic of interest to the research
try it
To explore further, ask a research question and write a hypothesis. Then create a survey of about six questions relevant to the topic. Provide a rationale for each question. Now define your population and create a plan for recruiting a random sample and administering the survey.

terms to know
Interview
a one-on-one, unstructured, long conversation between the researcher and the subject
Population
A defined group serving as the subject of a study.
Quantitative Data
Represent research collected in numerical form that can be counted
Qualitative Data
Comprise information that is subjective and often based on what is seen in a natural setting.
Random Sample:
A study’s participants being randomly selected to serve as a representation of a larger population
Samples:
Small, manageable number of subjects that represent the population
Surveys
Collect data from subjects who respond to a series of questions about behaviors and opinions, often in the form of a questionnaire.
Survey Research
A research method where subjects respond to the researcher's questions directly in an interview or questionnaire.
Focus Group
A group of individuals brought together in one room for a guided discussion of some topic of interest to research.

4. Secondary Data Analysis

While sociologists often engage in original research studies, they also contribute knowledge to the discipline through secondary data analysis. Secondary data analysis is the examination of existing data that have already been collected by other researchers. Secondary data do not result from firsthand research collected from primary sources, but are the already completed work of other researchers. Sociologists might study works written by historians, economists, teachers, or early sociologists. They might search through periodicals, newspapers, or magazines from any period in history.

Secondary data analysis is related to the literature review (step two of the scientific process: review the work of others), but it is not the same thing. A literature review is a step that comes before your own research, and which helps you to better refine and plan your study. You're seeing what other people have already done, in order to build your bibliography and demonstrate that you know what you're talking about when you position your research.

This is different from a secondary data analysis. In a secondary data analysis, a researcher uses existing data and sources as the center of their research, not just as informative background. A researcher might comb through giant data sets from the government, schools, or medical records to identify trends and relationships. This kind of work would be very time consuming and expensive if the data didn't already exist in some form. It also provides access to data that cannot be reproduced, from historical sources or studies.

EXAMPLE

The U.S. Census Bureau and the statistical abstracts of the United States are both secondary sources commonly used by sociologists. People like to use census data because it is scientifically collected, free and easy to access at Census.gov. Most importantly, it is valid and reliable data that is perfect for researchers to mine.

The use of secondary sources is also the primary methodology of historical sociology, which is a look at past societies and past structures to elucidate contemporary issues. Historical sociologists will often make use of library archives and secondary sources. The idea is to extract information from ‘A’, collect research from ‘B’, gather some insights from ‘C,’ then combine them to generate unique and new interpretations of social life and social phenomena that occurred in the past. In this way, secondary sources are a vital part of sociological research.

Using available information not only saves time and money but can also add depth to a study. Sociologists often interpret findings in a new way, a way that was not part of an author’s original purpose or intention.

EXAMPLE

To study how women were encouraged to act and behave in the 1960s, a researcher might watch movies, television shows, and sitcoms from that period. Or to research changes in behavior and attitudes due to the emergence of television in the late 1950s and early 1960s, a sociologist would rely on new interpretations of secondary data. Decades from now, researchers will most likely conduct similar studies on the advent of mobile phones, the Internet, or social media.

Social scientists also learn by analyzing the research of a variety of agencies. Governmental departments and global groups, like the U.S. Bureau of Labor Statistics or the World Health Organization, publish studies with findings that are useful to sociologists. :

EXAMPLE

A public statistic like the foreclosure rate might be useful for studying the effects of the 2008 recession; a racial demographic profile might be compared with data on education funding to examine the resources accessible to different groups.

One of the advantages of secondary data is that they are nonreactive research (or unobtrusive research), meaning that they do not include direct contact with subjects and will not alter or influence people’s behaviors. Unlike studies requiring direct contact with people, using previously published data doesn’t require entering a population, with all the investment and potential risks inherent in that research process.

Using available data does have its challenges. Public records are not always easy to access. A researcher will need to do some legwork to track them down and gain access to records. To guide the search through a vast library of materials and avoid wasting time reading unrelated sources, sociologists employ content analysis, applying a systematic approach to record and value information gleaned from secondary data as they relate to the study at hand.

But, in some cases, there is no way to verify the accuracy of existing data. :

EXAMPLE

It is easy to count how many drunk drivers are pulled over by the police. But how many drivers are not pulled over? While it is possible to discover the percentage of teenage students who drop out of high school, it might be more challenging to determine the number who return to school or get their GED later.

Another problem arises when data are unavailable in the exact form needed or do not include the precise angle the researcher seeks.

EXAMPLE

The average salaries paid to professors at a public school are a matter of public record. But the separate figures do not necessarily reveal how long it took each professor to reach the salary range, what their educational backgrounds are, or how long they have been teaching.

big idea
The advantages of secondary data analysis are that it can be cheaper and quicker to conduct than original research, and there aren't any concerns about ethical issues or approvals. The disadvantages are that some types of records may be difficult to access, it might be difficult to verify the accuracy of existing data, data might be available in a less useful form than a researcher might need, or the data may have been tainted by the biases of the original researchers or the time and place in which they were doing their work.

term to know
Secondary Data Analysis
The examination of existing data that have already been collected by other researchers.
Historical Sociology
Examining past societies and past structures to elucidate contemporary issues.


5. Combining Methodologies

These research methodologies are not mutually exclusive; many studies will combine these and other tools in order to best answer the research question. Combining questionnaires and interviews, or experiments and secondary research, are typical approaches to answering complex questions.

big idea
All research methods have advantages and drawbacks. Sociologists ideally let their research questions dictate the methods they're going to use. Some questions are best answered with experiments; some are better answered using surveys. Letting the questions dictate the methods helps the researcher to stay objective, and in fact, sociologists frequently use multiple methods at the same time.

In Context

Sociologists Devah Pager, Bruce Western, and Bart Bonikowski wanted to examine discrimination in the low-wage job market. They recruited white, Black, and Latino “testers,” who were assigned equivalent résumés and who were matched on a variety of characteristics such as age, education, physical appearance, and interpersonal skills. The testers were college-educated men who were organized into field teams that included a white, a Latino, and a Black tester. Their role as testers was to test the job market -- they were the independent variables, brought in to elicit a response from employers.

The testers applied to real job openings and recorded responses from employers. Because all testers were sent to the same firms, and testers were matched on a wide variety of characteristics, anything that might distinguish one applicant from the other was eliminated. The Latino testers spoke in unaccented English and were U.S. citizens of Puerto Rican descent and claimed no Spanish language ability, so that the difference between the testers would be only that of perceived race, and not other elements of their backgrounds.

They also examined the effect of a criminal record (felony drug offense) for different racial groups in job applications, building upon Pager’s research in 2003. Some applications included a checkbox to indicate a felony conviction and also listed prison labor as part of the applicant’s employment history. The teams applied for 340 real entry-level jobs throughout New York City over nine months in 2004.

In the results of the study, Black applicants were only half as likely as whites and Latinos to receive a callback or job offer, and whites, blacks, and Latinos with clean criminal backgrounds were no more likely to receive a callback as a white applicant just released from prison -- that is, a Black tester with no criminal record was at more of a disadvantage with employers than a white tester who did have a criminal record. However, the testers did not perceive any signs of clear prejudice in their job interviews.

As with many of the most insightful sociological studies, Pager, Western & Bonikowski included qualitative data based on the testers’ interactions with employers, which provided a rich supplement to the empirical data acquired through this field experiment. This multi-method approach uses a field experiment to collect empirical data, while also using testers' narratives of their interactions with employers as interpretive data.

Skill Reflect
Consider the research methods you explored in this lesson. How might you be able to use this knowledge to solve problems in your professional and personal life? Or, more importantly, how might you be able to review sociological research, with these methods in mind, and use researcher conclusions to address problems?

summary
In this lesson you learned about the principal research methods that sociologists use to collect quantitative data: experiments, survey research, and secondary data analysis. You learned about the advantages and drawbacks of each of these methods and how researchers select which one to use, and when it makes sense to be combining methodologies. Finally, you further strengthened your problem solving skills by analyzing these methods.