Peer Review Vs ‘Poor’ Review – Can a Systematic Plan Ensure Quality?

I was feeling a little disgruntled after in spite of two rounds of reviews, a reputable journal turned down a recent research effort of mine. I couldn’t help but disagree with many of their reviewers’ comments (I believe is a common sentiment among authors)– especially when some of them appeared to be very superficial and abstract. However, having reviewed for quite a few of the prominent and “high impact” cardiology journals myself, it eventually made me pause and think if I had been guilty of the same on occasion in the past. That led me to look up best practices for peer reviewing a manuscript, specifically for a cardiology journal. However, I did not have any significant success on locating such “guidelines.”

Peer review is largely considered to be a noble responsibility of a researcher, and considered an imperative skill for junior investigators.  I tried to come up with some semblance of a protocol for myself to save time in future peer review endeavors.

First and foremost, comes the decision to actually accept the peer review. In this day of mushrooming journals and inconsistent quality of manuscripts submitted even to the best of them, the decision to volunteer for a peer review or to decline respectfully is of paramount importance.  I personally would decline a review if either the subject matter is not of significant interest to me, or there is significant strain on time for the period allocated for the review by the journal. Of interest in the process of this decision to me also is the evaluation of potential conflicts of interest either declared or undisclosed by the authors. Such conflicts may directly arise from financial relationships of the author(s) to the subject matter of the manuscript – and often times from familiarity of the author with a ‘nominated’ peer reviewer as a professional colleague and/or a friend. Once I decide to review, the first piece of the manuscript that comes across is likely to be the abstract. Abstracts often are a window into the body of the manuscript – and merits close scrutiny. After all, most readers will likely read the abstract first as well. Needless to say that a quality manuscript should be able to invoke interest as well as provide evidence of scientific rigor even within the constraints of the word counts of the abstract. Simultaneously novelty of the investigation should well be portrayed through their abstract.

Next would come the introduction – this is supposed to lay the groundwork for the research details that follow subsequently in the script. However based on my own anecdotal experience, this is one of the least scrutinized segments for a peer reviewer in a rush. However, it may help to convey the logic as well as indicate prior work in the same area as the paper under consideration. That may even be unfamiliar to a seasoned peer review of the topic and may well be an educational treatise.

Then follows the methods section. Some degree of training and even mentoring can significantly help with the review of this section in my opinion. Journals should consider providing training in the various aspects of evaluating the methods section prior to enlisting a peer reviewer. Often times the journals do have statisticians/statistical consultants on their team/editorial board – however, imparting specific training for a volunteer peer reviewer who is considered an expert in a specific area of interest can potentially identify fatal conceptual errors which might otherwise be missed. For my purposes as a junior outcomes researcher without significant statistical training or expertise, I would recommend a statistician to review any part of a analysis plan that does not appear congruent.

Next for evaluation is the heart and soul – the results section of a manuscript. Of particular importance at this stage is to consider discarding/editing any redundancy – in the form of text, and/or figures and/or tables. Of great help to authors in improving a manuscript may stem from a reviewer’s suggestion of replacing any or all forms of the texts in the results with appropriate figures, and or tables with modifications of existing ones. More figures and tables may improve the readability of the manuscript as well.

Then comes the discussion section and it is here that the reviewer should decide if there is a thorough and balanced discussion of the results as reported in the previous section.  Evaluation of  references and adherence to the journal’s formatting criteria may have interest. Throughout the review process, help from a software to check spellings and grammar are of importance – may convey to the reviewer the lack of care and attention to details from the authors if there are too many.

Finally, enumerating ‘major’ vs ‘minor’ deficits may help overall evaluation. At the end of the peer review, there is the significant task of recommending a decision in the form of acceptance or revisions or rejection out right.  In most instances, the authors have dedicated significant effort and time – and deserve a fair and thorough evaluation leading to the decision.

The rewards for a detailed peer review are often a thank you note from the journal/editors, and more recently, CME credits have been a welcome addition. Some journals also list the peer reviewers in special issue. One idea that has been hotly debated for some significant time is the thought of having financial remuneration for peer review work – the idea being that reviewers would work as paid consultants to a journal. I don’t know how that may impact the quality of the process, but it may attract more interest upfront.

What are your thoughts?