“Masks are incapable of containing the transmission of COVID-19, concludes a study by USP” , reads the headline of an online magazine. It was reproduced by several anti-vax influencers without scientific training, including former president Jair Bolsonaro . The study mentioned in the article was published in the scientific journal BMC Public Health , but it has several methodological flaws. The magazine even added a note to the article warning that “the reliability of the data reported in this article is currently in dispute”.

The study “confirms” an idea dear to the millions of people who denied scientific discoveries during the pandemic, advocated early treatment and, like the authors of the article, opposed the use of masks. The article follows scientific methodology, but applies it based on an implausible hypothesis (masks would be harmful to the transmission of Covid), with an unreliable database (mask use based on self-reporting). The conclusions only confirm the authors’ initial bias. The study’s findings were denied by the fact-checking agencies Aos Fatos , Estadão Verifica and Lupa .

Recently, another article with relatively obvious methodological flaws, published in the Brazilian Journal of Psychiatry , “showed” that mediumship has a genetic basis. A quick search for the authors’ names shows that almost all (or all) of them follow the spiritist religion. The results obtained are the result of the authors’ biases and desires, not of nature.

A study published by a group at the University of California Irvine “showed” that essential oil diffusers used for seven nights (the Kabbalah’s favorite number) improved memory and cognition in the elderly. This would be revolutionary if it were true. However, the study also has several methodological flaws and its results are only taken seriously by companies that use it as evidence to sell essential oils, misleading their customers.

These examples show that not all science is created equal: there is high-quality science and there is bad science . But how can the average citizen, without any training in science, distinguish between the two, especially if he or she is presented with a published scientific paper (which, to the layman’s eyes, already gives authority to a scientific finding) that confirms his or her own beliefs?

To answer this question, it is necessary to understand how the validation of scientific ideas works.

Science, an indispensable tool for understanding nature

Science is, above all, a way of understanding nature. Despite being very recent in the history of humanity , science’s success in interpreting how and why things are the way they are has made it an indispensable tool for validating ideas about practically everything around us, as well as for creating technologies, from the simplest to the most sophisticated, that lead to a better quality of life and economic gains.

As a scientific community was created, it became clear that communication between scientists was essential to advance knowledge. Thus, scientific journals were founded, in which scientists publish their findings in the form of scientific articles , a very specific textual genre that allows the dissemination of ideas and the construction of new knowledge based on previously established knowledge.

Over time, quality control mechanisms have been created for the publication of scientific articles. It is not enough to have a new idea; authors need to convince editors — generally scientists with a high reputation among their peers — that their ideas correspond to the facts. One of the processes that seeks to ensure quality is peer review , in which an article submitted for publication receives the opinion of experts to assist the journal editor in making the final decision.

This process is very important, because scientists are human and their work is subject to errors, biases and mistakes of various kinds. Serious scientific journals, with rigorous review processes, have gained an impeccable reputation in the sense that we know that articles that have passed their peer review processes probably contain good ideas and accurate interpretations of nature.

Biases and beliefs clouding science

Due to the success of science in interpreting nature, many people seek validation of ideas and beliefs through it, even though, at times, these ideas and beliefs are not good or do not correspond to the facts.

Articles based on pseudoscience (when the research did not follow scientific methodology to be carried out, but the text is written in a way that seems scientific) or poorly done science (when the research followed scientific methodology but with errors — which can be serious or subtle — that lead to conclusions that do not correspond to reality) generally do not make it through the review process in rigorous journals. As a result, thousands of scientific journals have been created with editorial rigor that ranges from almost nonexistent to lax.

This has been going on for decades and has never been a major problem for scientists. But recently, these unreliable journals have been used by laypeople as irrefutable proof of the truth. Perhaps the clearest example of this is the COVID-19 pandemic, which has whetted the general public’s appetite for scientific information to an unprecedented degree .

When everyone wanted to understand what was happening and how it would be possible to avoid falling victim to the disease, some leaders — such as then-US President Donald Trump and Brazilian President Jair Bolsonaro — used poor-quality science to impose on their followers an anti-scientific interpretation not only of the behavior of the disease but also of the treatment and effectiveness of vaccines.

Activists for so-called “early treatment,” always politically aligned with these presidents, brandished “scientific” articles published in journals, claiming the wonders of chloroquine, ivermectin, and other drugs. None of these articles were published in highly prestigious journals. The seminal article on chloroquine treatment — now retracted by decision of the company that owns the journal (and not by the journal’s editor) —, for example, was published in a journal of low scientific rigor, whose editor was hierarchically subordinate to the main author.

Evidence of bad science

There are several indications that a scientific finding, even when published, may not have the rigor expected of science. Obviously, the main one is the quality of the journal in which the article was published. Articles with revolutionary or very important results generally appear in journals with high scientific impact and high rigor. I can mention Science or Nature , as well as other highly prestigious journals in different areas of knowledge. However, this rule is not absolute: very important articles may appear in less important journals.

Another sign of bad science is exaggerated or very surprising conclusions, such as the articles on the use of masks and COVID-19 and on essential oils, at the beginning of this text. In them, scientific methodology was applied sloppily, intentionally or not, arriving at the results that the researchers wanted instead of understanding the facts.

In general, articles with weak or inadequate methodology are the result of the authors’ beliefs or biases. A common example is authors with strong religious beliefs who seek, in some way, to demonstrate or prove that there is scientific evidence for their convictions, such as the article on mediumship that we cited.

It is worth remembering that poorly written scientific articles may have authors from renowned institutions, and their defenders invariably try to validate their dubious results through the prestige of the institutions. But one should not confuse the authors of poor-quality science with the institutions that host them. The University of California , USP and Harvard remain top-tier institutions, even if some of their researchers have published poorly written science.

Scientific consensus

Experienced scientists understand enough about methodology to know when a paper, even if it has been published, is of poor quality. That is why scientific consensus is essential. Consensus consists of the informed opinion of the scientific community. When many scientists read a paper and agree that it is poorly done, then it is most likely poorly done.

To find consensus, it is worth looking for opinion pieces on the websites of renowned institutions or highly prestigious magazines, or even following the specialized press. Another reliable source is fact-checking agencies, such as those already mentioned, which have a reputation to maintain, study in depth and seek out authoritative opinions from real experts rather than biased “scientists” or activists.

Our understanding of nature is based on good science. Thus, we know that masks used correctly most likely reduce the spread of respiratory diseases, essential oils do not improve memory in the elderly, mediumship has no genetic basis, and chloroquine does not cure COVID-19. If a well-written paper in the future shows otherwise, the scientific consensus will certainly change.

Author Bio: Leandro R. Tessler is Professor at the Glab Wataghin Physics Institute part of the State University of Campinas (Unicamp)