«

»

Apr
21

Testing reviewer veracity – Proposing new ways to ensure your grants and papers are read

Veracity – 1. truthfulness, honesty. 2. Accuracy. (The concise Oxford Dictionary, 1990).

Life imitates art and art imitates life, not a single day goes by without some form of inspiration coming from the rest of the scientific community. Whether this is people emailing, tweeting, blogging or reviewing papers or grants.

I know we are increasingly short of: time, attention spans (and money) but it just amazes me how poorly people do what they were supposed to do.

If you sign up for reviewing a paper, read it. If you sign up for a grant, read it. And I do not mean just the abstract in both cases.

Clearly a recent paper in DDT slipped in review. My guess is the reviewers either did not read it or where associated with the author. There appear to be so many red flags that this paper is just plain iffy to begin with. (And I am on the editorial board of this journal).

How often does this happen that bad papers get published..probably it’s pretty rare (?), but has anyone actually bothered to assess this? I would argue it is also likely good papers have issues with poor reviews because they are just not read. Yes there is an expectation that the author make the manuscript clear and understandable, but shouldn’t the reviewer commit to reading the whole thing?

So how do we check this? I was inspired a bit by security guards and how they do the rounds in buildings. They literally have some bar code or some other point to check that they actually visit buildings or specific offices on their rounds. What if there was some way to check that a reviewer actually read the whole paper. Could someone devise a method in which the reader scrolls down a document then has to complete some codes to unlock more of the document to ensure they actually read the sections before they could submit the review. How could we do this so that it was not something that could be sidestepped or gamed?

Technically its probably very doable. Should it be done? Yes IMHO because so much depends on it. Grants can be from tens of thousands of dollars to tens of millions of dollars..So much rides on publications in journals, as these are ultimately of value for getting grants, promotion, tenure etc. Even if scientists say they have no “conflicts of interests with the author or grant submitter, they are at the whim of other pressures for their time and that ultimately has to reflect directly in the quality of the reviews they give.

So yes we need to ensure reviewer veracity, it can probably be done pretty cheaply and simply. Decades of experience in some cases have shown us we can no longer rely on honor for peer review, actually reading a document matters.

 

Leave a Reply

Your email address will not be published.

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>