A constant stream of health and wellness information can make it hard to know which advice to take.
Staying informed is important to staying healthy. But not every new research study trending on Twitter is scientifically valid.
The next time a splashy health story blows up your Facebook feed, consider using these questions to judge the credibility and validity of research. They’ll help you judge which studies are scientifically sound – and what’s safer to ignore:
How large is the sample size?
The sample size is the number of subjects (people, mice, bacteria, etc.) the researchers studied. An appropriate sample size depends on the design of the study, but generally, larger sample sizes are better than smaller ones. The larger sample size more accurately reflects the larger population from which it was drawn. Also, different results in different subjects within a small sample can be explained as chance, or coincidence. It’s harder for sheer chance to influence a larger sample.
For human studies there are several guidelines you can use. If a research study compares two influencing factors (i.e., variables) there should be a minimum of 30 participants in each group in the study. If there are multiple factors (i.e., multivariate analysis) being studied within a larger body of participants there should be closer to 200 participants at a minimum.
Was there a control group?
Remember this from 4th-grade science fair? A control group is the part of the study sample that did not receive the experimental treatment, medicine, food or experience. Medical studies usually treat a control group with whatever is the standard type of care for a certain illness or problem, compared with other sample groups that get the new treatment the research is studying. Credible research includes a control group, because it allows researchers to compare the results of the experimental treatment with what happens in its absence.
Is the study from a credible group, or endorsed by one?
This holds true not only for studies, but for websites you might read when researching anything from nutrition to vaccines. For websites, read the “About Us” section. Who owns the website? What is the owner’s agenda or mission? Is it explained clearly? Where does this group or person get its funding? That will tell you a lot about the website’s goals. Simply put, is the source credible and easily identified?
Generally, the medical community in the U.S. considers the following organizations, among others, scientifically sound sources of health information:
- American Academy of Pediatrics
- American Academy of Family Physicians
- American Cancer Society
- National Institutes of Health
- National Cancer Institute
- Centers for Disease Control and Prevention
- Medical Library Association
- U.S. National Library of Medicine
- Guideline.gov, a national clearinghouse of clinical practice guidelines based on scientific evidence
What journal published the study?
Anthropologist and geneticist Jennifer Raff explains in her blog, Violent Metaphors, that reputable biomedical journals will be indexed by Pubmed, part of the National Institutes of Health; and by Web of Science. A report in a widely respected journal is more credible than something cited on a company brochure, for example.
Was the research reviewed by other scientists?
Respected scientific journals have a panel of experts who read the articles to judge their validity before accepting them for publication. A reputable journal may reject an article for many reasons that suggest the research study’s findings aren’t sound.
Have the results been duplicated? Can they be?
The methods section of a research article should explain the steps researchers followed, so someone else can duplicate the study and, presumably, get the same results. If other researchers carry out the same experiment but get different results, that may cast doubt on the initial study or it may call for further investigation.
Is this a lab study, or one conducted on people?
Research findings based on lab work are much further from becoming clinical treatments than those already being studied in people.
How can I learn to read the scientific paper?
If you want to read an actual study report – rather than news coverage of it – you’ll probably need some help navigating these highly technical documents.
Raff provides this thoughtful primer on how to understand scientific papers.
A key piece of advice here from Raff: “Reading a scientific paper is a completely different process than reading an article about science in a blog or newspaper. Not only do you read the sections in a different order than they’re presented, but you also have to take notes, read it multiple times, and probably go look up other papers for some of the details. Reading a single paper may take you a very long time at first.”
But feeling confident in the health advice you’re following is worth the effort.
For more tips on reading scientific papers and news coverage, check out these guides from Family Caregiver Alliance and the U.S. National Library of Medicine.
No matter what health trend is in or out, research over decades has consistently found that the best way to protect your health is to eat a balanced, nutritious diet; exercise regularly; and avoid smoking.
