A Brief Introduction to Common Criteria for Evaluative Indexing Services

a-brief-intro-duction-to-common-criteria-for-evaluative-indexing-services

Viv Gorayska, Digital Licensing Controller, Wiley

September 05, 2019

One of the most closely-watched milestones for a journal is acceptance into bibliographic databases such as Web of Science (WoS), Scopus, and PubMed (either MEDLINE-indexing or PMC Full Participation). Coverage in these products provides a public stamp of approval, enhances discovery, drives traffic, attracts authors, and may (in some cases) lead to the infamous Impact Factor. As Digital Licensing Controller, I oversee Wiley’s relationship with the organizations that run these databases, and I have worked with many editors and Wiley colleagues over the years to facilitate applications.

While each service has its own acceptance criteria and evaluation workflow, there are several common standards. This article is therefore necessarily general and not exhaustive; not all points may be relevant for your journal, but it should serve as an introduction to the main areas of examination. Rejections will lead to a block on re-applying for at least two years, so we should do as much as we can to ensure a journal meets the criteria from launch and at least before putting it forward for consideration.

Subject Coverage

Both Web of Science and MEDLINE are highly selective, with 10% and 12-15% acceptance rates respectively. Their focus is to serve customers/users with comprehensive, quality subject coverage. Any new titles will therefore be judged against what is already indexed, with reviewers looking for novelty and high impact content. Scopus and PMC are less selective, with a greater emphasis on breadth of coverage and sound science but with similar rigor in applying their quality standards.

Ethical Policies

There are two parts to the evaluation of ethical policies. Firstly, organizations will check if the journal has publicly stated its policies, normally via the author guidelines on the journal website. The evaluators should take the type of publication into consideration when appraising policies, to make sure that they are appropriate for the articles published.

For example, if a journal doesn’t publish primary research then it doesn’t need to have a policy covering the use of animals in experiments, or if no research conducted with humans is published then there is no need to obtain informed consent. All journals are expected to have a requirement for declaring potential conflicts of interest. It would also be beneficial to state in the author guidelines whether the journal conforms to recognized standards such as the Declaration of Helsinki, the ICMJE Recommendations, or the EuCheMS Ethical Guidelines for Publication in Journals and Reviews, if appropriate.

Once the existence of policies has been established, the evaluators will check whether the policies have been followed. This is normally done by reviewing published articles to see if consideration of research ethics is discussed in the text, if Institutional Review Board approvals are mentioned anywhere (if applicable), and that conflicts of interest are declared (including null declarations). Bear in mind that users of WoS and/or PubMed will be accessing the journal at the article level, so this is where reviewers would expect policy compliance to be evident.

Wiley has a standard template for author guidelines which can be referenced to ensure that what is available for your title is appropriate and comprehensive. Further information can also be found in Wiley’s Ethics Guidelines Research Ethics in Journal Articles section.

Peer Review Policy

All the major evaluative services require that journals be peer-reviewed to meet their minimum criteria, at least for all articles reporting on primary research. The peer review policy should be clearly available on the journal website and, at a minimum, cover the blinding-level and whether special handling measures are in place for submissions by editorial board members, to eliminate bias. If relevant, adding a description of a manuscript transfer program would be useful; details on recommending reviewers and an appeals process can be included if desired.

Editorial Board

The main administrative check on the editorial board is for the presence of members’ affiliations on the website and, for MEDLINE, also qualifications.

Other points that will be analyzed are the geographical spread of the editorial board, with international representation favored for non-regional titles. The size of the editorial board should reflect the volume of articles published, and the members’ expertise should match the scope of the journal.

Publication records of the editorial board will be checked. Web of Science, being concerned with citation analysis, will look at the citation networks of the editorial board members, as well as checking if there is a disproportionate number of citations to the journal coming from the editorial board.

Authorship

The geographic and topical range of authorship should reflect the editorial board. Variety in authorship will be expected, with papers coming from several sources; repeat-authorship, with multiple papers by the same authors within a short time period, is frowned upon. Also of concern are papers from editorial board members, who should not be over-represented compared to the size of the field, as this will imply narrow appeal and a small author pool. Again, Web of Science will examine the authors’ citation profiles.

Other Qualitative Indicators

Wiley receives feedback, both negative and positive, from indexers following an evaluation, which allows us to pick up on other areas that are of interest to them. Recent feedback has included comments on reproducibility of primary research, judged by the information in the methods sections, and also clarity of language.

As part of Wiley Editing Services we offer English language editing and translation support to authors as they are preparing their article, with a major expansion of our author services offerings planned by the end of 2019.

Web of Science will check the self-citation rate of a journal during the evaluation (and will continue to monitor it after acceptance). The size of the subject community and the scope of the journal will be considered, but it is advisable to keep the rate below about 30% if possible.

Conclusion

Some of the evaluation criteria can be esoteric and opaque, and final decisions can appear subjective. But you can help your journal’s chances, or at least prevent damage to them, by taking some practical steps. Increase the transparency of the journal’s organization and policies wherever possible, show that research ethics are taken seriously, and make sure to put your best foot forward in terms of style and language.