Peer Review, Science, and the Stars

Peer ReviewYou should be dubious of everything you read on this blog, almost everything on the Web, and in most popular press because there is no peer review of the content.  Sure, some human other than the writer looked over the content before somebody else hit the Publish button, but that is not peer review.  Indeed, if you think that peer review is just human review, you’re not thinking clearly about words and how people use them.  Tell me the other common, ordinary uses of the term, “peer review?”

No other fields beyond scientific, academic, and scholarly fields use the term or the process.  Thus, even though the phrase is composed of ordinary words that would simply denote “somebody else reads it,” the phrase is not in ordinary parlance and thus should alert you to a special meaning.

Peer review is a brutal process of anonymous content review from selected readers in your field of knowledge.  Before an editor even picks up a red pencil or that annoying red highlight function in Word, your paper is sent to several people with an obvious and well earned reputation in your field.  These proven, tough, and experienced researchers then anonymously read your work and provide comment to both you and the editor.

The first review is frequently fatal.  You receive a polite and unmistakable drop dead letter from the editor with an attachment of the anonymous reviews.  Sometimes, you are allowed to rewrite your paper in light of the anonymous review and resubmit for re-review.  Thus, you get to do the whole thing all over and hope you revised smartly into the criticisms of the anonymous reviewers.  Rarely, about ten percent of the time with better journals, you get your report accepted.  What then follows is the kind of “review” that occurs with most publication sources:  An editor red pencils the text, legal looks it over for lawsuit potential, and maybe marketing suggests how to position it.

Nothing like peer review happens at the New York Times or Newsweek magazine or CBS News or Huffington Post or Penguin Press.  The closest is that review by inhouse editors who read the paper largely for style and style sheet, but rarely for content accuracy.  Any content concerns usually go to a lawyer to discover copyright or libel issues.  But, a content review, a pure content review?  Nope.  Not an industry standard.

So what?  Big deal?  Who cares?  Peer review, bah humbug.

I’ve been in the peer review machine since 1984.  I’ve written for and reviewed for a wide variety of peer review journals in communication, psychology, and health and safety.  If you can trust just one guy’s experience, I assert that virtually every paper I reviewed and rejected for a peer review journal was better than most things I read in the popular press.  The quality of the writing and thinking was at least as good as every opinion column in the New York Times or the Wall Street Journal.  And, these papers were the ones that got rejected and never saw the light of day in the peer review literature.

Now, is this to claim that peer review always insures the Truth, the Whole Truth, and Nothing but the Truth?  Of course not.  The peer review process publishes errors, lies, manipulations, mistakes, brilliant dead ends, and all manner of less than the Truth.  Look up the name “Cyril Burt.”  Go read the original epidemiological research on hormone replacement therapy.  Search on “bad blood” and the CDC.  Read now the East Anglia emails on climate change.  People do bad things, dumb things, harmful things and manage to get it through peer review.

Like anything made or done through human activity, it could be more efficient or more effective, not because the thing made or done is inherently inefficient or ineffective, but because people possess a human nature that is not perfect.  However, peer review can be nicely analogized with Winston Churchill’s line about democracy:  It is the worst form of government except for all others that have been tried.  Peer review is the worst form of scientific communication we’ve got except for all others.

The advantage to peer review is that eventually other voices are heard.  I would suggest that for almost any writer it would be easier to get published in a scientific journal than to get published in the New York Times or the Washington Post or any elite popular press outlet.  In both cases, you have to master a style sheet, a content area, and express yourself well, but with pop press, if you are not close to the Cool Table, you haven’t a ghost of a chance.  With peer review, you have more than a ghost of a chance.  It may require persistence, but virtually everything in life requires that.  Except getting published in popular press.

Peer review provides the best chance for True science.  Anyone who criticizes that process as a villain in any controversy is making an error in judgment.  No better replacement for peer review waits in the wings and no evidence points to an inherent weakness in peer review that caused the villainy.  That failure lies . . .  Shakespeare anticipates us:

“The fault, dear Brutus, is not in our stars,
But in ourselves, that we are underlings.”