When it comes to complex, scientific findings, credibility is a must — which is why post publication peer review is becoming more common.

It’s no surprise that the Internet can be filled with sketchy, sometimes half-baked theses, often based more on assumptions than fact. But in a sea of noise and falsehoods, there are certain factors we all should look for in an attempt determine validity.

One of these trust factors is peer reviewed publication. The process of peer review, during which a work is evaluated by two or more experts in a field, is considered an essential part of scientific publication — but that doesn’t mean it’s always perfect.

Peer review: the good and the bad

Traditional peer review allows experts to scrutinize and comment on work before it’s officially published and released. The works quality is assessed as thoroughly as possible, and only the highest quality work will be accepted for publication.

Even so, peer review has it’s flaws:

• Reviewers names and comments are typically not published
• With only several reviewers, mistakes can be made
• There is often limited transparency on review processes
• Experts in periphery fields, or opposing ones, are generally not involved

Even the most respected journals have ended up publishing flawed or even fraudulent content in spite of rigorous peer review. In one experiment, a paper which deliberately included eight mistakes was sent to over 200 reviewers. The average of mistakes called out? Less than two.

Enter the Internet

It’s worth noting that the number of reported ethical problems in papers rose from one per year before 1999, to over 50 a year after 2004 when most publications were made public online. This is likely because the Internet helps reviewers detect potential issues that could have been missed in the past.

This just goes to show the potential of technology and the web for revolutionizing peer review, spotting plagiarism, and detecting image falsifications. The mode for peer review has changed to reflect this electronic shift, but the model is, so far, much of the same.

One alternative that has been tested is open peer review, through which reviewers of all backgrounds transparently review authors’ work over the Internet.

A trial of this process in Nature, however, concluded that not only were researchers and scientists reluctant to offer open comments, buy the comments offered were not substantive and the idea unpopular among many authors.

Wisdom of the crowd

Another novel concept the Internet has brought to science and research is what some have called the “wisdom of the crowd,” which is exemplified in the constantly-updated and ideology-free resource Wikipedia.

Similarly, Google’s algorithms are able to rank importance by incoming links from other sites, among other factors essentially determined by the wisdom of the web, while websites like Reddit measure content based on upvotes and ranking, and involve extensive discussion and input by and for users.

Because the volume and pace of information and voices on the web is so extreme, it is difficult for static, closed works to keep up. Is there a better way? It’s a question worth asking, and one for which some are posing solutions.

Post publication peer review

Pre-publication peer review may indeed have room for improvement, but most agree that it’s a process best suited for talented experts that graciously devote their time and knowledge to the betterment of their field, free of cost.

But a new concept of post publication peer review is emerging, and proving an illuminating trend that can keep research accountable, in focus, as well as spark online discussion. Essentially, such services augment peer-reviewed work with an extra layer of review from a new type of peer: Internet folk.

The most prominent of these is PubPeer, a website that allows open comment by scientists and experts on any published scientific article available. Authors are then notified of comments and invited to respond and discuss. Comments are required to use facts that can be publicly verifiable.

Already, PubPeer users have identified flaws in work that has led to subsequent retraction, correction, and scrutiny.

But even PubPeer is not without its critics. In one case, an author sued PubPeer commenters and demanded their names be released, to varying success. Some also worry that open comments can escalate quickly into vicious attacks, a phenomenon anyone that has ever visited an online forum can attest to.

The future of peer review

Post publication peer review is growing. As PubPeer gains traction, similar endeavors are abound. These include:

• PubMed Commons: Enables published authors to share opinions and information on published PubMed articles.
• Open Review: allows registered members to upload formal and transparent reviews of published works
• SynValuate: Platform for the crowdsourced review of chemical synthesis

New technology and the Internet are making it possible for experts to have discussions about and analyze published work like never before.

Though the nature of the web may make this uncomfortable, disorganized, and even threatening for some, at the end of the day — so long as the pursuit of knowledge is the goal — adjusting to trends that take advantage of the Internet’s many minds and voices will be a good thing for science and society.