It is obvious that having the best, most qualified, most diligent reviewers is desirable both for guiding editor’s decisions and giving authors feedback. Finding the best reviewers, however, can be a real challenge. Editors, being experts in their fields, will already have a number of contacts in their areas that they can call on, but with submissions increasing editors can rarely rely on a limited reviewer pool.

Source: Forest Woodward/iStockphoto

1. Build a database
One important element is to have a strong reviewer database in place. The pool of reviewers is something that is built up over time, but structuring and maintaining that database is essential. Asking reviewers to supply keywords for their areas of expertise, (where possible for a taxonomy recognized in the field) provides a significant advantage when trying to match papers to reviewers. Rating reviewers within the database, on timeliness and quality (for instance), can provide a useful metric for identifying better reviewers, as can metrics regarding turnaround times and completion rates. Flagging the record of slow and unhelpful reviewers means they can be avoided in future.

2. Ask the experts

To find new reviewers, it’s important to solicit ideas from the journal’s editorial board. These are individuals who are on the board in recognition of their expertise, so they’re both possible reviewers and aware of other potential reviewers in the field. Similarly, it can be useful to ask for suggestions for alternatives from those reviewers who decline to review.

3. Mine article references
Another method of finding new reviewers is to mine a submission’s references for the authors of related works. There is no guarantee that an author will be a suitable reviewer just because one of their articles is cited, but looking at their other publications and their research interests should give a better estimation of their suitability. As well as looking at references, editors can also look at the authors of their journal’s articles. Because these authors have already had their work scrutinized by the journal, the editors have some degree of surety of the expertise of these individuals.

4. Beware of fakes
Recently, a number of high profile cases (here, here and here for example) have highlighted the danger of unscrupulous authors attempting to act as reviewers on their own papers. The problem is caused by the option many journals offer for authors to suggest reviewers. Either the author themselves, or a third party service acting on their behalf, creates an email account and purports that it is the email address of their suggested reviewer. If the editor chooses to invite that suggested reviewer, then the invitation goes to that email account which is, in fact, accessible to the author.

Editors should already be cautious about being dependent on reviewers suggested by authors because of the temptation for authors to suggest only those who will be sympathetic to their research. The danger of pseudonymous reviewers should make editors doubly cautious about suggested reviewers.

One telltale sign of pseudonymous reviewers is the use of free email services, such as Yahoo, Hotmail or Gmail, as opposed to a reviewer institutional address. A simple web search is usually sufficient to find or validate a reviewer’s institutional email address.

5. Trust your instincts
Incidents such as these are a warning against an overdependence on automation. Electronic systems can provide invaluable aid in managing and searching databases but they cannot replicate the insight of an editor.