Reading the Evidence
Preprints, Peer Review and What Each One Guarantees
Peer reviewed is treated as a stamp of truth and preprint as a warning label. Both descriptions overstate what the process actually does.
Comparisons of peer review and publication processes usually pick a winner. This one picks the circumstances, which is more useful.
The difference in one place
- Peer review is a limited check by a small number of unpaid readers.
- Reviewers rarely see raw data and are not auditing for fraud.
- Preprints trade scrutiny for speed and are not automatically unreliable.
What peer review involves
An editor sends a submitted manuscript to a small number of researchers working in a related area. Those reviewers read it in their own time, unpaid, alongside their own work, and return comments within weeks. They assess whether the methods appear appropriate, whether the conclusions follow, and whether the writing is clear.
They usually work from the manuscript alone, without access to the raw data or the original records. It is a useful filter performed by busy people with partial information rather than a verification of results.
What it does not do
It does not confirm that the experiments were performed as described or that the numbers are correct. It rarely detects deliberate fabrication, which typically surfaces later through inconsistencies noticed by other readers. It does not guarantee the finding will replicate, which is a separate matter requiring independent repetition.
Where the claim is carefully worded, it also does not mean reviewers agreed with the conclusions, only that the paper cleared the threshold for publication. Treating publication as a verdict rather than as an entry into the conversation misunderstands the whole system.
Preprints and what they change
A preprint is a manuscript posted publicly before or alongside submission to a journal. It makes findings available immediately, which matters enormously during fast-moving public health situations.
Against the trial data, it has not been through review, so errors that reviewers would have caught may still be present. Preprints are read and criticised publicly, which is a different form of scrutiny rather than an absence of it. Many are subsequently published with little change, and some are substantially revised or never published at all.
Journals that publish anything
A category of journals charges publication fees while performing minimal or no genuine review. They frequently have plausible-sounding names and rapid acceptance times measured in days. A citation to such a journal looks identical to a legitimate citation in a marketing footnote.
Where the claim is carefully worded, checking whether the journal is indexed in major databases and whether the editorial board is real takes a few minutes.
This is one of the few places where checking the container matters as much as reading the contents.
Retraction and its long tail
Papers are retracted when errors or misconduct are established, and the retraction is recorded formally. Retracted papers continue to be cited for years afterwards, because citations propagate faster than corrections. A claim circulating widely may rest on a paper that no longer stands, and the circulation rarely mentions it.
On the label, databases exist that flag retracted work, and checking one is a reasonable step before relying on a striking claim. The existence of retraction is a sign the system corrects itself, slowly and imperfectly rather than not at all.
A practical checking routine
Find the actual paper rather than the article about it, since abstracts are freely available for almost everything. Read the methods before the conclusions, because the methods determine what the conclusions can support.
Check the journal, the funding declaration and the conflict of interest statement, all of which are usually printed. Look for whether anyone independent has replicated the finding, which matters more than where it appeared. None of this requires specialist training, and it takes less time than reading the news coverage would.
Side by side
| Consideration | What it means in practice |
|---|---|
| What peer review involves | Peer review is a limited check by a small number of unpaid readers. |
| What it does not do | Reviewers rarely see raw data and are not auditing for fraud. |
| Preprints and what they change | Preprints trade scrutiny for speed and are not automatically unreliable. |
The takeaway
Peer review is a limited check by two or three busy readers. Judge a paper by its methods and its replication, not by its badge.
The body already has organs for this, and none of them are sold in a box.
Questions readers ask
Should I distrust preprints?
Not automatically, since many are perfectly sound and peer review is a limited check anyway. Treat both with the same questions about methods, sample size and independent replication.
Does publication in a famous journal mean it is right?
Prestigious journals publish work that later fails to replicate, like every other journal. The venue is weak information compared with the design of the study itself.
