This tutorial will explain how data is collected and how to analyze data critically. This tutorial will specifically focus on:
When you look at published data, it is important to look at it critically. You have to think about the who, what, where, when and why of the situation.
Were there problems in defining or measuring the variables of interest? Did they have problems with trying to figure out what was going to be measured or how to measure it? For whom was the data gathered? Was this done for an independent agency or was this done for a company? Is the company that funded it trying to push a product? When was it gathered?
Another important question settles on relevancy. Is it out of date? If it's out of date, then maybe you don't want to use it, because maybe there's something a little bit more recent, and a little bit more reliable out there.
Why did they do it? What are they trying to prove? They might be having an agenda here. And were any biases present in the study? And these might be biases that the researchers overlooked.
In the academic world, many people will examine and judge the study prior to it being published. If no major flaws are found, then that's evidence of a well-done study. Basically, the more eyes you can get on the study, the better. This is a form of quality control, and it's called peer review.
What it means is you take other individuals from within your discipline, and have them critique your study. The majority of scholarly articles are peer-reviewed prior to being published. Once published, more people can weigh in on what they see.
It may seem really harsh to critique someone's work like this, without having done the experiment yourself. But ultimately it may prompt a follow-up experiment, and the feedback provided will make the subsequent research more well done.
When multiple people examine studies, and they do it critically. The study undergoes intense consideration by experts in the field. This is called peer review. The collected data is run through a battery of questions by others in the field such as “why was this information gathered?” When multiple experts look at the study, it may prompt further experiments or they may catch errors that require fixing. If no issues are found, then it is evidence of a well-done study.
Good luck!
Source: this work is adapted from sophia author jonathan osters.
