If you don't remember your password, you can reset it by entering your email address and clicking the Reset Password button. You will then receive an email that contains a secure link for resetting your password
If the address matches a valid account an email will be sent to __email__ with instructions for resetting your password
Department of International Medical Center/Ward of General Practice and National Clinical Research Center for Geriatrics, West China Hospital, Sichuan University, Chengdu, China
Department of International Medical Center/Ward of General Practice and National Clinical Research Center for Geriatrics, West China Hospital, Sichuan University, Chengdu, China
Department of International Medical Center/Ward of General Practice and National Clinical Research Center for Geriatrics, West China Hospital, Sichuan University, Chengdu, China
Department of International Medical Center/Ward of General Practice and National Clinical Research Center for Geriatrics, West China Hospital, Sichuan University, Chengdu, China
Corresponding author: Department of Periodical Press and National Clinical Research Center for Geriatrics, West China Hospital, Sichuan University, Chengdu 610041, China. Tel: +86-199-83137949; fax: +86-28-85582944.
Corresponding author: International Medical Center / Ward of General Practice and National Clinical Research Center for Geriatrics, West China Hospital, Sichuan University, Chengdu 610041, China. Tel: +86-18-980601359; fax: +86-28-85582944.
Department of International Medical Center/Ward of General Practice and National Clinical Research Center for Geriatrics, West China Hospital, Sichuan University, Chengdu, China
We conducted the first retrospective cohort study to evaluate the impact of copublication on the citation of Cochrane evidence across a wide range of Cochrane review groups.
•
Copublication of Cochrane reviews in other journals was associated with a higher citation frequency for the evidence and might increase the impact factor of the copublishing journal.
•
Copublication might facilitate broader application of Cochrane evidence and promote its dissemination.
•
Authors may wish to give more consideration to copublishing their Cochrane reviews in another journal.
Abstract
Objectives
To evaluate the impact of copublication on the citation of Cochrane evidence.
Study Design and Setting
This was a retrospective cohort study including Cochrane reviews published up to 31 December 2015 and their citing up to 11 July 2021, identified from the Web of Science Core Collection database.
Results
A total of 101 copublished and 202 noncopublished Cochrane reviews were included. The median for the total number of citations and the medians for the numbers of citations to the Cochrane review in the first, second, third, and fifth years after publication in the copublished group were higher than those in the noncopublished group [71 (interquartile range {IQR}: 37.5, 118.5) vs. 32.5 (13, 67); 1 (0, 3) vs. 0 (0, 1); 6 (3, 11.5) vs. 2 (1, 5); 8 (4, 15) vs. 3.5 (1, 8); 8 (4, 15) vs. 3 (1, 9), respectively, all P < 0.001]. Copublication of Cochrane reviews meant that 4 of 21 journals and 6 of 22 journals had a higher impact factor in the first and the second year after the copublication than they would have had without the copublication.
Conclusion
Copublication is associated with a higher citation frequency of Cochrane reviews and may increase the impact factor of the journal in which it is copublished. This facilitates broader application of Cochrane evidence and promotes its dissemination.
This of 101 copublished and 202 matched noncopublished Cochrane reviews found that the median for the total number of citations and the medians for the numbers of citations to the Cochrane review in the first, second, third, and fifth years after publication in the copublished group were higher than those in the noncopublished group.
What this adds to what was known?
•
Copublication of Cochrane reviews in other journals was associated with a higher citation frequency for the evidence and might increase the impact factor of the copublishing journal.
•
Copublication might facilitate broader application of Cochrane evidence and promote its dissemination.
What is the implication and what should change now?
•
Authors of Cochrane reviews may wish to give more consideration to copublishing their reviews in another journal.
1. Introduction
Since 1994, Cochrane (previously The Cochrane Collaboration) has allowed copublication of Cochrane reviews in specialist journals in certain circumstances [
]. Copublication of Cochrane reviews is not a duplicate publication or academic misconduct but a way to present the findings and conclusions of the Cochrane review to the readers of a specialist journal to improve access to the Cochrane evidence. The journal version is a secondary publication of the Cochrane review, using the data and interpretation of the version published in the Cochrane Database of Systematic Reviews (CDSR) [
]. Copublication of Cochrane reviews might be as an abridged version of the full review, in a “Cochrane corner” of a journal (where the summary of the review is accompanied by a commentary) or a short version translated into a language other than English [
]. Adherence to the formatting requirements of the copublishing journal and the Cochrane copublication policy are required for copublication and the copublished version should be peer-reviewed and edited as per the journal's editorial process. Journals interested in arranging copublication of a Cochrane review need to sign an agreement with Cochrane and, as of July 2021, 82 specialist journals had done so [
]. The Cochrane Collaboration intended that the copublished version would meet the same quality standards and would allow the implications of the review to be applied more broadly and would increase dissemination of the information [
]. Copublication of a shorter version of the Cochrane review might also help those readers who felt challenged by the full-length review. As an extreme example, the Cochrane review with the largest number of included studies (as of May 2022) is a network meta-analysis of drugs to prevent postoperative nausea and vomiting and, if printed, this would fill more than 2,200 pages [
With so many potential benefits of copublication, it might be expected that copublication would be common. However, for example, only 19.6% of reviews from the Cochrane Eyes and Vision Group were copublished in 2007 and 16.2% in 2014, showing that fewer than one in five of the reviews from this Cochrane review group were copublished and that there was no increase over a decade [
Identification and characterization of systematic reviews in eyes and vision. Presented at the XV Cochrane Colloquium, 2007 Oct 23–27, Sao Paulo, Brazil.
]. The reasons for this were unclear but might include a lack of awareness of copublication among journal editors, a reluctance by authors to spend time on copublication (especially if the copublished content requires significant changes), a lack of a consensus between Cochrane and other journals on copublication, and the need for different formats [
Thus, it is important to study the impacts of copublication. The first descriptive study explored the characteristics of copublication in the Cochrane Eyes and Vision Group [
]. Wang et al. found that the number of citations for copublished reviews was approximately twice than that of the Cochrane Eyes and Vision Group's other reviews. However, the generalizability of this beyond this one of approximately 50 Cochrane review groups was not known. Therefore, we performed the present study to assess the impact of copublication of Cochrane reviews across all Cochrane review groups. We investigated this in two ways: citations of copublished vs. noncopublished Cochrane reviews and the impact on the IF of the copublishing journal.
2. Methods
2.1 Study design and search strategy
This was a retrospective cohort study of Cochrane copublication based on data collected from the Web of Science Core Collection database. We report it as per the Strengthening the Reporting of Observational Studies in Epidemiology guideline [
], 44 were indexed in this database. We searched the database from inception up to 31 December 2015 using the term “Cochrane” and the names for these 44 journals to identify eligible reviews and up to July 11, 2021 to obtain data on the numbers of citations. The search strategy is shown in Appendix 1. Cochrane reviews which were copublished at the same time or later in journals were the cases (copublication group), although the controls were noncopublished Cochrane reviews from the same Cochrane review group which were published in the same year as the copublished Cochrane review (noncopublication group). We used a case to control ratio of 1:2.
2.2 Inclusion and exclusion criteria
Inclusion criteria were (1) the copublished review was published after the publication of the full Cochrane review in the CDSR and (2) the copublished review was based on the Cochrane review published in the CDSR. Exclusion criteria were (1) copublication was a duplicate report; (2) Cochrane reviews that had been updated because the publication that would otherwise have been used; (3) copublication was an editorial, letter, or conference abstract; and (4) a sufficient number of matching Cochrane reviews that had not been copublished was not available.
2.3 Random sampling
We used stratified random sampling with proportional allocation within each stratum, where the targeted variables were Cochrane review groups and year of publication of the Cochrane review which had been copublished [
]. First, the sampling strata were the 27 Cochrane review groups with at least one copublished Cochrane review and the 16 publication years (from 2000 to 2015). The Cochrane review groups are listed in Appendix 2. Specifically, all Cochrane reviews published by a Cochrane review group in each year in which they had published a copublished Cochrane review were identified from CDSR. We then excluded any reviews that had been subsequently withdrawn or updated or had been copublished in a journal which was not among the 44 journals with signed agreements with Cochrane which were indexed in the Web of Science Core Collection database. The remaining candidates formed the sampling frame and, within each stratum, we randomly sampled reviews in a 1:2 ratio to match copublished and noncopublished Cochrane reviews. If there were not at least two eligible noncopublished Cochrane reviews for each copublished review, the review was excluded. The sampling procedure was carried out by one author (Y.Z.).
2.4 Review selection and data extraction
Decisions about eligibility and data extraction were undertaken independently by two authors (L.L.Z. and Y.W.). Disagreements were resolved by consensus with arbitration by another author (R.Y.) if necessary. We used a standard form to independently extract the following data for eligible reviews: article title, authorship of CDSR and journal version, country of first author, journal of copublication, timing of copublication in relation to publication of the Cochrane review, number of included studies, results and conclusions, and type of copublication. The number of citations to the Cochrane reviews and to the copublished versions and the IF of the copublishing journal were taken from the Web of Science database (Journal Citation Report).
2.5 Outcomes
This retrospective cohort study aimed to examine the potential impact of copublication on the promotion and application of evidence-based practice that is based on Cochrane reviews. The primary outcome measures were the mean and median for the total number of citations to the Cochrane reviews and the numbers in the first, second, third year, and fifth year after publication. The secondary outcomes were the ratio of an adjusted IF (excluding the data for the copublished Cochrane review) and the actual IF of the copublishing journal. These two indicators were used to evaluate the impact of copublication on the copublished journal.
The journal IF is calculated by dividing the number of citations in the current year by the number of articles published in the previous 2 years, which we derived from the Journal Citation Report of Web of Science [
]. The formula for the IF means that a simultaneous increase/decrease in the number of articles and citations can lead to IF variations in either direction, depending on which variable changes most [
]. To determine whether the copublications affected the journal's IF, we created an adjusted IF by, first, subtracting the number of citations for the copublished review from the total number of citations (i.e., A) and subtracting the number of copublished reviews from the total number of published articles (i.e., B), A divided by B gives the adjusted IF, which was the IF that the journal would have achieved if it had not copublished the Cochrane review. We then calculated a ratio by dividing this adjusted IF by the journal's actual IF. If this ratio was more than or equal to one, the copublished reviews did not increase the journal IF, although values less than one indicated that the copublished reviews increased the journal IF. If a journal had published more than one copublished review, we calculated a mean ratio of IF and its 95% confidence interval (95% CI) to account for the multiple articles.
We calculated the number of citations for each copublished review as the sum of the citations to the Cochrane review and to its copublished version, which were derived from the Web of Science database.
2.6 Statistical analysis
We used the Pearson’s Chi-squared test to compare categorical variables, the Student’s t-test for continuous and normally distributed variables, and the Mann-Whitney U-test to compare medians if the data were not normally distributed. A P value of <0.05 was considered significant. We used SPSS 25.0 software for the statistical analyses.
3. Results
3.1 Baseline characteristics of Cochrane reviews
Our search for “Cochrane” and the name of one of the 44 eligible journals identified 1,442 publications from the Web of Science Core Collection database up to December 31, 2015, containing 171 possible copublications of Cochrane reviews. After we applied our inclusion and exclusion criteria, 101 of these were eligible for the study. We then randomly sampled 202 of the noncopublished Cochrane reviews in accordance with the methods described above to obtain the 1:2 ratios of cases and controls. The flow diagram is shown in Figure 1. The included reviews were from 27 Cochrane review groups and had been published between 2005 and 2015. The first authors of the copublished reviews and the noncopublished reviews were from 14 countries and 28 countries, respectively. The main characteristics of the two groups of review are shown in Table 1. No significant difference was found in characteristics between the two groups.
Fig. 1Flow diagram for identification of the cohort of Cochrane reviews used in this study.
In considering any copublication of a Cochrane review (not just those in the 44 journals with signed agreements with Cochrane), we retrieved a total of 8,717 Cochrane reviews from the Web of Science Core Collection database up to December 31, 2015. After removal of duplicates, 6,830 Cochrane reviews remained and we identified copublications for 148 of these between 2005 and 2015. The annual copublication rate in this period fluctuated from 0.96% to 3.94% and there was no clear trend over time (Fig. 2).
Fig. 2Annual co-publication rate of Cochrane reviews (2005 to 2015).
The 101 copublished Cochrane reviews were copublished in 22 of the 44 journals which signed copublication agreements with Cochrane. Twenty eight (27.7%) were published in the journals in the same year as the Cochrane review was published and more than three-quarters of copublications (79.2%) were published within 2 years after the publication of the Cochrane review. The types of copublication were mostly short sections of the Cochrane review (such as the abstract or plain language summary) or an abridged version of the full review (64.4%), with smaller but similar proportions for specially prepared summaries of the review (14.9%) or a “Cochrane corner” (15.8%). The copublication content was usually based on the original Cochrane review (76.2%), with the number of included studies (in 85.1% of the reviews) and the results and conclusion (88.1%) being the same in both versions.
Only 58 (57.4%) of the copublications clearly contained all the content required in the Cochrane copublication policy [
], among which six (10.3%) did not place it on the title page. Although a further 24 (23.8%) of the copublications contained relevant information for the corresponding Cochrane review, it was not fully in accordance with the requirements. Characteristics are summarized in Table 2.
Table 2Characteristics of 101 copublications (journal version)
Short version of a Cochrane review: this includes republishing a part of the review (such as the abstract and plain language summary) or an abridged version [19].
This requirement is that a footnote should be placed on the title page of the journal version to inform readers and documenting agencies that the article has been published previously, either in whole or in part, citing the primary publication, for example, “This article is based on a Cochrane Review published in the Cochrane Database of Systematic Reviews (CDSR) YYYY, Issue X, https://doi.org/10.1002/14651858.CD00xxxx (www.cochranelibrary.com provides information). Cochrane reviews are regularly updated as new evidence emerges and in response to feedback and the CDSR should be consulted for the most recent version of the review.” [2].
Other places included the background, introduction, conclusion, or acknowledgment section of the article.
6 (10.3%)
Did not meet the requirements for copublication content
24 (23.8%)
No detailed description of whether met the requirement
19 (18.8%)
Copublication based on
Original review
77 (76.2%)
Updated review
24 (23.8%)
Number of included studies
Same as Cochrane review
86 (85.1%)
More than Cochrane review
7 (6.9%)
Less than Cochrane review
8 (7.9%)
Results and conclusion
Same as Cochrane review
89 (88.1%)
Different results and conclusion
7 (6.9%)
Different results, same conclusion
5 (5.0%)
a Short version of a Cochrane review: this includes republishing a part of the review (such as the abstract and plain language summary) or an abridged version [
d This requirement is that a footnote should be placed on the title page of the journal version to inform readers and documenting agencies that the article has been published previously, either in whole or in part, citing the primary publication, for example, “This article is based on a Cochrane Review published in the Cochrane Database of Systematic Reviews (CDSR) YYYY, Issue X, https://doi.org/10.1002/14651858.CD00xxxx (www.cochranelibrary.com provides information). Cochrane reviews are regularly updated as new evidence emerges and in response to feedback and the CDSR should be consulted for the most recent version of the review.” [
3.3 Number of citations for copublished and noncopublished Cochrane reviews
The median for the total number citations up to July 2021 was significantly higher in the group of copublished reviews than in the noncopublished group [71 (37.5, 118.5) vs. 32.5 (13, 67), P < 0.001]. Similarly, the median number of citations in the first year, second year, third year, and fifth year after publication in the copublished group was higher than in the noncopublished group [1 (0, 3) vs. 0 (0, 1), 6 (3, 11.5) vs. 2 (1, 5), 8 (4, 15) vs. 3.5 (1, 8), 8 (4, 15) vs. 3 (1, 9), respectively, all P < 0.001] (Table 3).
Table 3Total number of citations for the copublished and noncopublished reviews
The 101 included Cochrane reviews were copublished in 22 journals, one of which did not have an IF until the second year after its copublication of a Cochrane review. Among these journals, the ratio of the adjusted IF to the actual IF was less than one in four (19.0%) of 21 journals for the first year after publication. In the second year after publication of the copublished review, the ratio was less than one in six (27.3%) of 22 journals. This means that the copublished review increased the IF for these four and six journals in the first and second year after publication of their version of the Cochrane review. The results for all journals are shown in Table 4.
adjusted IF: Total number of citations—number of citations for the copublished review/Total number of published articles—number of copublished reviews.
Colorectal disease did not have an IF until 2007 and the copublished review was published in 2005.
0.980
Developmental Medicine & Child Neurology
0.997
0.992
Diabetic Medicine
1.001 ± 0.002 (0.994-1.008)
0.998 ± 0.001 (0.996-1.001)
European Journal of Physical and Rehabilitation Medicine
1.015 ± 0.003 (1.009-1.021)
1.009 ± 0.004 (1.001-1.016)
International Journal of Nursing Studies
1.001 ± 0.002 (0.996-1.006)
1.002 ± 0.002 (0.997-1.006)
Journal of Pediatrics and Child Health
1.003
1.003
Journal of Rehabilitation Medicine
0.995
0.998
Journal of Rheumatology
1.005 ± 0.001 (1.002-1.008)
1.005 ± 0.001 (1.002-1.008)
Journal of the American Academy of Dermatology
1.002 ± 0.001 (0.995-1.008)
1.001 ± 0.001 (0.994-1.007)
Revista Portuguesa de Cardiologia
1.006 ± 0.001 (1.002-1.011)
1.008 ± 0.002 (1.002-1.013)
Spinal Cord
0.999
0.993
Spine
0.999 ± 0.001 (0.998-1.001)
0.998 ± 0.001 (0.996-1.000)
Stroke
1.001
1.001
Techniques in Coloproctology
0.993 ± 0.011 (0.853-1.132)
0.995 ± 0.009 (0.880-1.109)
a Colorectal disease did not have an IF until 2007 and the copublished review was published in 2005.
b If there was only one result (IF ratio) for a journal, we could not calculate an SD and 95% CI.
c actual IF: Total number of citations/total number of published articles.
d adjusted IF: Total number of citations—number of citations for the copublished review/Total number of published articles—number of copublished reviews.
If the ratio of the two IF is less than 1.00, this indicates that the inclusion of the copublished Cochrane review led to a higher IF for the journal than would otherwise be the case.
4. Discussion
The present study was designed with the aim of exploring the potential impact of the copublication of Cochrane reviews. To our knowledge, this is the first retrospective cohort study investigating copublication that covers multiple disciplines and multiple Cochrane review groups. Our results show that the number of citations for copublished Cochrane reviews was significantly higher than that for noncopublished reviews both in total citations and in citations in the first 5 years after publication. In addition, the IF for four of 21 journals was raised by the publication and citation of a version of a Cochrane review in the first year after its publication and in six of 22 journals in the second year. Together, these findings suggest that copublication of Cochrane reviews not only improved the dissemination and accessibility of Cochrane evidence but also facilitated its uptake.
For the period of our study, the proportion of Cochrane reviews that were copublished varied year on year (0.96% to 3.94%), with no clear trend over time. These proportions were well below that found for one individual Cochrane review group [
Identification and characterization of systematic reviews in eyes and vision. Presented at the XV Cochrane Colloquium, 2007 Oct 23–27, Sao Paulo, Brazil.
]. Although some authors and editors might be concerned that copublication was an academic misconduct or duplicated publication, this was not the case. As per the International Committee of Medical Journal Editors (ICMJE) recommendations, duplicate publication is publication of an article that overlaps substantially with one already published, without clear, visible reference to the previous publication [
]. Moreover, ICMJE has proposed a conceptual framework of ‘acceptable secondary publication’. Specifically, secondary publication or copublication is permitted as a means to disseminate important information but several conditions need to be met [
], such as approval from the editors of both journals, an intention that the secondary version is for a different group of readers, and a clear indication that the secondary version is based on an article that has been published elsewhere. On this basis, Cochrane developed a series of policies to promote copublication [
], which are consistent with the guidance of ICMJE, and therefore, in keeping with good practice.
We have shown that the number of citations of copublished Cochrane reviews was significantly higher than those of noncopublished Cochrane reviews, which is consistent with a previous study [
]. This increased number of citations suggests benefits for the Cochrane evidence. First, each copublishing journal had a specialist audience, thereby bringing the findings of the Cochrane review to that audience [
]. Second, the higher number of citations suggests that the corresponding Cochrane review would have received more attention because of the copublication. Third, the copublished version might make the Cochrane evidence more accessible to potential users than the full Cochrane review and might be more useful to them as a reference source [
This was the first study which examined the impact of copublication on the copublishing journals. It investigated the effect of the copublication on the journal's IF and found that four of 21 journals had a higher IF in the first year after publication and six of 22 journals had a higher IF in the second year because of the copublished Cochrane review. The longer-term effects were less clear because of the way in which the IF was calculated but citation rate peaks vary across different journals, ranging from 2.5 to 7.2 years after publication [
Current concepts on bibliometrics: a brief review about impact factor, Eigenfactor score, CiteScore, SCImago Journal Rank, Source-Normalised Impact per Paper, H-index, and alternative metrics.
]. This is an area for further research. Furthermore, it has been acknowledged that comparing IF across different areas of research might not be meaningful [
] and, because the 27 Cochrane review groups with data in this study represented a wide range of areas, the size of any impact on a journal's IF might vary for different disciplines.
This study showed that Cochrane's copublication policies [
], including description of copublication type and placing of a footnote on the title page of the journal version were not fulfilled by all authors. Among the group of 101 copublished reviews, only two-thirds (68, 67.3%) of the copublications explicitly cited the original Cochrane review and only 58 (57.4%) of the copublications provided the information in a footnote on the title page of the journal version. Citation of the corresponding Cochrane review and a prominent position of this information are important for showing readers of the copublished articles to find the original Cochrane review.
Our study has some limitations. First, we only incorporated data on citations and copublications from the Web of Science Core Collection database. Second, some Cochrane reviews published an updated version in a later year, which we excluded from this study, which might lead to selection bias. Third, there might be selection bias between specific copublications and noncopublications but this should not affect our results or conclusions because journals and editors have a variety of priorities for publishing articles. Fourth, although some studies have found that meta-analyses have higher rates of citation than other articles, which might make it more likely that copublishing a version of Cochrane containing meta-analyses would have more impact on a journal's IF than copublishing any other type of article; other studies have argued that study design does not significantly influence citation rate [
]. These issues should be investigated in further research.
In conclusion, we have shown that copublication increases the number of citations for Cochrane reviews and may increase the IF of the copublishing journal. This facilitates a broader application of Cochrane evidence and promotes its dissemination. On this basis, we encourage Cochrane authors to copublish their reviews and we believe that copublication may be a “win-win-win” for practitioners, journals, and patients.
CRediT authorship contribution statement
Linlin Zhu: Formal analysis, Investigation, Visualization, Writing – original draft. Yu Zhang: Methodology, Visualization, Data curation. Rong Yang: Methodology, Software. Yang Wu: Validation, Investigation. Yi Lei: Resources, Data curation. Yonggang Zhang: Conceptualization, Project administration, Supervision, Writing – review & editing. Xiaoyang Liao: Project administration, Supervision. Mike Clarke: Project administration, Supervision, Writing – review & editing.
Identification and characterization of systematic reviews in eyes and vision. Presented at the XV Cochrane Colloquium, 2007 Oct 23–27, Sao Paulo, Brazil.
Current concepts on bibliometrics: a brief review about impact factor, Eigenfactor score, CiteScore, SCImago Journal Rank, Source-Normalised Impact per Paper, H-index, and alternative metrics.
Conflicts of interest: Mike Clarke is the co-Editor-in-Chief of the Journal of Evidence-Based Medicine and Yonggang Zhang is the journal's managing editor, who has a copublication agreement with Cochrane. All authors declared there was no other potential commercial, professional, or other conflicts of interest relevant to this article.
Funding: This work was supported by Sichuan Science and Technology Program (No. 2019YJ0102).
00138-X&id=gr1.jpg){kind=link}
00138-X&id=gr2.jpg){kind=link}