Quality of Scientific Evidence

A fundamental aspect of effective science communication is recognizing the varying quality of scientific evidence. Not all evidence holds the same weight.

Expert knowledge is the least reliable form of justification. While it can be valid, it only holds merit in the absence of reasonable counterarguments, which is often not the case.

Case reports and case series rank slightly higher. These documents detail individual or grouped cases that exhibit similar phenomena. For instance, if a patient with a fever takes aspirin and subsequently recovers, this observation is recorded as a case report, while multiple such experiences would constitute a case series.

Next, we have animal studies, which can provide useful insights but are inherently flawed as models. The biological differences between animals like mice or rats and humans may render the findings inapplicable to human subjects. While these studies are ethically more favorable, their ability to inform human health can be limited, particularly when relying on simplistic models, such as C. elegans, which, although beneficial for basic neuroscience, lack relevance for more complex cognitive functions.

Cross-sectional studies are effective for gathering population data and identifying correlations, but they fall short in establishing causation.

Case-control studies improve upon this by comparing individuals with and without a condition to determine possible exposures to causative factors. Although they are cost-effective, their findings may not be as reliable.

On the other hand, cohort studies are prospective, tracking individuals over time based on their exposure to potential causes. This approach reduces the risk of random chance affecting the results.

Randomized controlled trials are regarded as the gold standard in clinical research due to their ability to limit confounding variables. However, they can be costly and may raise ethical concerns, particularly regarding withholding established treatments.

Finally, systematic reviews and meta-analyses are often viewed as the pinnacle of research because they synthesize findings across multiple studies. They differ from other research types as they analyze existing studies rather than investigate new phenomena.

When evaluating research, it’s essential to understand that some studies are inherently stronger than others. For example:

• Systematic reviews outperform individual studies.
• Human studies are more reliable than animal studies.
• Experimental studies provide more robust evidence than observational ones.
• Blinded studies are superior to non-blinded studies.
• Larger studies yield more reliable results than smaller ones.
• High statistical significance is preferable to low significance.

Scientific justification can vary in robustness, with some theories being more substantiated than others. The ideal scenario is a systematic review demonstrating consistent, high-quality studies supporting a theory, representing the strongest justification available.

Although this discussion primarily addresses medical research, the principles apply across various scientific domains, including physics and chemistry, where systematic reviews can aggregate and analyze findings effectively.

Proper Word Choice

While discussing evidence quality, it’s crucial to remember the nature of science itself. In science, theories cannot be confirmed or proven true, a dilemma known as "the problem of induction" that has been debated since the times of Hume and Popper. Therefore, terms like "confirmed," "proven," and "fact" should be avoided in science communication. Even without the problem of induction, uncertainty persists, emphasizing the importance of assessing evidence quality.

Sensationalism can be mitigated by avoiding such terms and adhering to the hierarchy of evidence. Media often sensationalizes discoveries with headlines claiming "the cure for cancer" or "the secret to immortality." Such language can misrepresent scientific findings, stemming from a lack of proper word choice and an understanding of evidence quality.

Citations

An article I came across recently discussed mercury levels in freshwater fish—wouldn't you want to read it? It’s frustrating when articles reference "a study" without proper citations. This issue is as vexing as sensationalism in science communication and often intertwined. When authors cite original studies, readers can verify the accuracy of the reporting.

In today’s digital age, hyperlinking to original studies is straightforward, and providing complete citations enhances transparency. While scholarly articles aren't always necessary, linking to sources early on and including full citations at the end is a best practice I advocate.

Citation Machine

Citation Machine is one of the most effective tools for generating citations in various formats. It can pull information from websites, scholarly articles, and books, offering both free and paid options.

doi2bib

This tool simplifies citation generation in BibTeX format from digital object identifiers (DOIs), which I frequently use when writing scholarly papers. BibTeX provides an efficient system for managing references.

BibTeX entry from URL

This extension generates BibTeX references for active URLs, making it user-friendly, though minor formatting issues may arise.

BibTeX to APA

Converting Google Scholar BibTeX references to APA format is possible with this tool, enhancing usability for those who need to adhere to specific citation standards.

Sometimes, I admit to being less meticulous when writing blog articles, not always citing every detail. Yet, nothing frustrates me more than an article claiming to cite a study without providing accessible references. Such practices fall short of good science journalism, which I’ve observed in various media outlets, including prominent ones like The New York Times. Linking to studies or providing citations should be straightforward, and it’s essential to avoid sensationalizing findings while ensuring a clear understanding of study results.

Summary

Ensuring proper citation of article sources is crucial. When referencing scientific papers, always link directly to them without burying citations within the text. Avoid sensational language like "proven" and "fact," and consistently evaluate the quality of the evidence presented.

Originally published at the Freelance Journalism Alliance of the Guild Association

This article serves as a concise overview of how to enhance science communication and journalism. The Freelance Journalism Alliance aims to use this knowledge, along with insights from various sources, to develop a series of MOOCs.