Who says romance is dead?! We have reached the half way point in this blog series, and our romance with Peer Review is far from waning, in fact it’s fair to say that we are still very much in the honeymoon phase.

We’ve already committed to being ethical and having integrity and here we move onto “Fairness”, another essential area of peer review practice, about which we share recommendations for journal teams in our article “What does Better Peer Review Look Like?” A dedicated group of Wiley colleagues published this work in the peer-reviewed journal Learned Publishing in January 2019. We first shared a longer version of the article in a preprint. Our goal from this work is to enable journal teams to identify areas where their practice is great and areas where they may want to make improvements.

This post explores aspects of “Fairness” in peer reviewed publishing with the help of Nigel Andrew, Managing Editor for the journal Austral Ecology, Elizabeth Brenner who is Associate Managing Editor for a portfolio of life, medical, and social science journals at Wiley, and Andrew Moore PhD, Editor in Chief of BioEssays, a peer-reviewed hypotheses and perspectives journal, and Editor for Biosciences of Advanced Science, a peer-reviewed primary literature journal at Wiley.

Fairness, we suggest in our article, requires that peer review “considers papers on their own merit, without regard for the identity of the author(s) or the reviewers’ and editors’ own interests… Fairness is also rooted in a straightforward moral axiom (‘treat others as you would like to be treated’).”

Q. Nigel, what do you think fairer peer review looks like?

A. If a manuscript is at either end of the quality spectrum, i.e., either accept, minor changes (grammatical) or reject outright, then the peer review process is relatively straightforward. The most important peer review comments, though, are needed for manuscripts that need “major revisions.” Here, authors need clear and focused explanations about why their manuscripts need major changes, and also enough positive feedback to warrant them putting in that extra effort to get it up to standard. This means making overall comments to start with, and then sharing a suite of specific requirements (line by line) so authors can easily agree (or not) to make the requested changes.

Q. Andrew, the same to you: what does fairer peer review look like?

A. For me, maintenance of confidentiality throughout and after one’s review of a manuscript, whatever model of peer review is adopted, is absolutely crucial to maintaining trust in peer review itself. Reviewers need to know they should not breach this confidentiality. On a related note: if a peer reviewer gives a very scathing peer review report of a manuscript in the first round, the same peer reviewer should be more willing to review it in the second round. (If you visit the article preprint, there is a case study where a first-round reviewer “bowed out” at this stage and recommended a similarly-minded peer to finish the job).

Q. Thanks, both. Andrew, your response is intriguing, in part because we are experimenting with giving authors the option to use a transparent peer review process, where peer review content is published (with a DOI) along with accepted articles (and optionally with reviewer signatures). I guess my take is that everyone involved should respect the conditions under which the article was submitted and the peer review invited. This is how the Wiley Review Confidentiality Policy is worded. Back to the questions, Elizabeth, what is your top priority for improving fairness in peer review?

A. The current conversation around peer reviewers is that they are hard for journals to find, even while many hundreds of early career researchers and practitioners are available and eager to review but presumably are not being asked. Editors are not sure how to cultivate these un-invited reviewers, perhaps wary of their ability to provide quality feedback, and so rely on networks of people they know and trust, who are often established and further along in their careers. I believe that changing the conversation to discuss how much power peer reviewers have in the process, and how much responsibility comes with that power, will help shift thinking in a direction of how to be ethically responsible when reviewing a paper, and not just how to offer adequate constructive criticism. It may be easier to have these conversations with up-and-comers who do not have years of ingrained assumptions and behaviors to challenge.

Q. Thanks for that, Elizabeth. This certainly rings true with research we’ve started (and will share later in 2019) into what early career researchers think about journal publishing. Elizabeth, what practical steps would you recommend?

A. Clear policies and instructions for reviewers and editors are a great place to start, but these instructions and policy statements are often only glanced at or skipped altogether as reviewers go directly to the task at hand. The next step is inserting questions directly into review forms, asking reviewers to acknowledge the journal’s policies and to disclose any conflicts. Many editors publish editorials on these topics and link authors and reviewers to those publications for guidance. An annual discussion on ethics for editor teams encourages exploration and leadership. Requiring the journal editorial board to read and sign conflict of interest disclosures and other materials laying out expectations is a good way to have this conversation with your core reviewers. Journals can offer a simple online training course or certification to new reviewers that covers both ethical standards and how to write a good review, and that course could be modified to speak to established reviewers who may be unaware of the current industry conversations around transparency and standards.

Thank you, Nigel, Elizabeth, and Andrew.

Journal team members who are interested in making their peer review processes better can read our article, published under a Creative Commons license by Learned Publishing. We have created a Better Peer Review Self-Assessment tool, derived from this work.

With our Better Peer Review Self-Assessment, you can record your reflections on your current practices, and you can plan new directions. You will receive an immediate summary of your results, then we’ll follow up with your total score by quartile, your Better Peer Review Badge, and Data Visualization. And there’s more: Our Better Peer Review Self-Assessment works if you’re based inside Wiley or outside Wiley at one of the many societies and associations for which we’re proud to publish. Ask your Wiley publisher for more information! We hope you find the whole Better Peer Review experience useful, and that you’re able to identify areas for new directions and improvements to peer review at your journal.