Advertisement

Poor compliance of clinical trial registration among trials included in systematic reviews: a cohort study

  • Kristina Lindsley
    Correspondence
    Corresponding author. Tel.: +1-443-381-2414; fax: +31-88-7568099.
    Affiliations
    Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, the Netherlands

    Cochrane Netherlands, University Medical Center Utrecht, Utrecht University, Utrecht, the Netherlands

    Life Sciences, IBM Watson Health, Baltimore, MD, USA
    Search for articles by this author
  • Nicole Fusco
    Affiliations
    Life Sciences, IBM Watson Health, Baltimore, MD, USA
    Search for articles by this author
  • Hannah Teeuw
    Affiliations
    Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, the Netherlands
    Search for articles by this author
  • Eva Mooij
    Affiliations
    Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, the Netherlands
    Search for articles by this author
  • Rob Scholten
    Affiliations
    Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, the Netherlands

    Cochrane Netherlands, University Medical Center Utrecht, Utrecht University, Utrecht, the Netherlands
    Search for articles by this author
  • Lotty Hooft
    Affiliations
    Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, the Netherlands

    Cochrane Netherlands, University Medical Center Utrecht, Utrecht University, Utrecht, the Netherlands
    Search for articles by this author
Open AccessPublished:December 14, 2020DOI:https://doi.org/10.1016/j.jclinepi.2020.12.016

      Highlights

      • Less than one-third of clinical trials in a sample of systematic reviews were registered.
      • Only 37% of registered trials were registered prospectively.
      • Only 31% of registered trials provided trial results with the registration record.
      • Some improvement in clinical trial registration recently, but still less than ideal.
      • More stringent enforcement of prospective clinical trial registration is needed.

      Abstract

      Objectives

      The objective of the study was to examine whether clinical trials that have been included in systematic reviews have been registered in clinical trial registers and, when they have, whether results of the trials were included in the clinical trial register.

      Study Design and Setting

      This study used a sample of 100 systematic reviews published by the Cochrane Musculoskeletal, Oral, Skin and Sensory Network between 2014 and 2019.

      Results

      We identified 2,000 trials (369,778 participants) from a sample of 100 systematic reviews. The median year of trial publication was 2007. Of 1,177 trials published in 2005 or later, a clinical trial registration record was identified for 368 (31%). Of these registered trials, 135 (37%) were registered prospectively and results were posted for 114 (31%); most registered trials evaluated pharmaceutical interventions (62%). Of trials published in the last 10 years, the proportion of registered trials increased to 38% (261 of 682).

      Conclusion

      Although some improvement in clinical trial registration has been observed in recent years, the proportion of registered clinical trials included in recently published systematic reviews remains less than desirable. Prospective clinical trial registration provides an essential role in assessing the risk of bias and judging the quality of evidence in systematic reviews of intervention safety and effectiveness.

      Keywords:

      Key findings
      • Less than one third of clinical trials in a sample of systematic reviews registered.
      • Only 37% of registered trials were registered prospectively
      • Only 31% of registered trials provided trial results with the registration record

        What this adds to what was known

      • Some improvement in clinical trial registration recently, but still less than ideal.

        What is the implication and what should change now

      • More stringent enforcement of prospective clinical trial registration is needed.

      1. Introduction

      Systematic reviews of randomized clinical trials produce the highest level of evidence for informing the effectiveness of health care interventions [
      • Guyatt G.H.
      • Sackett D.L.
      • Sinclair J.C.
      • Hayward R.
      • Cook D.J.
      • Cook R.J.
      Users' guides to the medical literature. IX. A method for grading health care recommendations. Evidence-Based Medicine Working Group.
      ]. The quality of evidence relies on the credibility of the trials included and whether the trials were likely to be at risk of potential bias. To minimize bias, the methods of trials should be outlined before conducting the trial, and deviations should be documented. Clinical trial registers allow trial investigators to prospectively register their intention to conduct a trial and the main methods and outcomes of the trial before enrolling the first trial participant.
      In addition to registering trials to minimize methodological biases and maximize transparency, there are also ethical implications. Research participants who volunteer and consent for their information to be used do so with an understanding that their participation will contribute to medical research and further scientific knowledge. If trials are not made known to the public and their results are not disseminated, the implicit agreement between the study participant and researcher is broken. Furthermore, this is a form of research waste which may result in duplicate studies being conducted to examine research questions which may already have been answered by previously conducted studies.
      In 2004, the International Committee of Medical Journal Editors (ICMJE) recommended that journals consider publishing articles reporting clinical trials of health care interventions only when the trial had been registered prospectively [
      • De Angelis C.
      • Drazen J.M.
      • Frizelle F.A.
      • Haug C.
      • Hoey J.
      • Horton R.
      • et al.
      Clinical trial registration: a statement from the international committee of medical journal Editors.
      ]. ICMJE recognizes six clinical trial registries in addition to 10 other primary registries included in the World Health Organization (WHO) International Clinical Trials Registry Portal (ICTRP). As of September 27, 2007, US law charges the US Food and Drug Administration (FDA) with overseeing clinical trial registration as a requirement for all ‘applicable clinical trials’ of drugs, biologics, and devices (FDAAA 801) [
      Food and Drug Administration Amendments Act of 2007.
      ]. The law requires that in addition to registering the trial before enrolling the first participant, the trial investigators are required to submit results for trials that investigated an FDA-approved drug, biologic, or device within 12 months of the completion date of the trial. Similar requirements for posting clinical trial results were outlined by the European Medicines Agency in 2014 [
      European Medicines Agency
      Posting of clinical trial summary results in European Clinical Trials Database (EudraCT) to become mandatory for sponsors as of 21 July 2014.
      ]. In 2013, the international AllTrials campaign was launched, calling for all trials to be registered and the results reported in accordance with the Declaration of Helsinki principles [
      AllTrials
      About AllTrials. All Trials Registered | All Results Reported.
      ]. Specifically, the campaign lists four areas of reporting for each trial: 1) registration, 2) summary of trial results in the same place as the registration, 3) details of study methods and results (e.g., full report in compliance with Consolidated Standards of Reporting Trials (CONSORT)), and 4) individual patient data, of which the first three areas should be made available in the public domain.

      2. Objectives

      The objectives of the study were to examine whether clinical trials that have been included in systematic reviews have been registered in clinical trial registers (e.g., ClinicalTrials.gov) and, when they have, whether results of the trials were included in the clinical trial register. We also assessed whether trial results published in journal articles were made available in the public domain (i.e., open access) and describe trial characteristics (e.g., year of publication, number of participants).

      3. Methods

      3.1 Data source

      We identified clinical trials from systematic reviews published by the Cochrane Musculoskeletal, Oral, Skin and Sensory (MOSS) Network from 2014 to 2019. We used Cochrane reviews because they are limited to clinical trials, which are the type of study design of interest for this project (i.e., studies required to be registered in a clinical trial registry) and they search trial registers in addition to bibliographic databases. The MOSS network includes eight topic-specific review groups: 1) back and neck; 2) ear, nose, and throat; 3) eyes and vision; 4) musculoskeletal; 5) oral health; 6) pain, palliative and supportive care; 7) skin; and 8) wounds.
      Eligible reviews were intervention reviews published within the past 5 years (September 2014 to September 2019) that included at least five trials (n = 618). Reviews that had been withdrawn, overviews of reviews, reviews of diagnostic test accuracy, reviews of prognosis, and protocols of reviews were not eligible. We used a random number generator in Microsoft Excel to select 10 reviews meeting the eligibility criteria from each of the eight review groups (except for Eyes and Vision, for which 30 reviews were selected as part of the initial pilot study). In all, 100 out of the 618 eligible reviews were included (citations of included reviews are listed in Appendix A).

      3.2 Data collection

      We designed and pilot tested a data extraction form in DistillerSR (Evidence Partners, evidencepartners.com) to collect data from each of the 100 randomly selected reviews. In addition to hierarchical data extraction (i.e., compatible for extracting data on multiple trials included in a single review), DistillerSR allows for serial review of data extraction. One person extracted data for each review, and a second person verified the data extracted. Any discrepancy between the two reviewers was resolved by discussion.
      Data collection included review characteristics, such as the condition under investigation, the type of interventions being examined, the number of included trials, whether meta-analysis was performed, and the review authors’ conclusions. We also collected data on the characteristics of each trial included in each review, such as when the trial was published, the number of participants randomized, the country of the trial, whether a trial registration record was reported by the review authors, whether the trial provided data for meta-analysis, and whether the full published study report was available open access.
      If no trial registration record was reported by the review authors and the trial was published in 2000, when ClinicalTrials.gov became publicly available, or more recently, we searched study reports and trial registers to determine if the trial was registered. In the first searching phase, we reviewed the abstracts of trial references and, when the full-text report was available open access, we searched the full report for a trial registration ID. If no trial registration ID was found from the study reports, we used condition and intervention terms to search the two clinical trial registry databases that are endorsed by the ICMJE: ClinicalTrials.gov (www.clinicaltrials.gov) and the WHO ICTRP (www.who.int/ictrp/en/). We confirmed trial registration matching by comparing the study investigators and/or institutions and sponsors, the number of participants, the study period, and the study design.
      When a trial registration record was identified, from either the review authors or our own searching, we recorded whether the trial was registered prospectively or retrospectively (registration submitted more than 1 month after study start date) and documented whether the trial results were posted within the trial registration record. We classified posted results as efficacy outcomes only, safety outcomes only, or both efficacy and safety outcomes. Acknowledging that some investigators may consider linking the trial registry record to a journal publication with trial results, we also assessed whether trial results published in full journal articles referenced by the review authors were available in the public domain; we searched PubMed, Google Scholar, and Google for an open access article or document (i.e., the full-text report was available free of cost).

      3.3 Data analysis

      We summarized descriptive statistics (medians, ranges, and proportions) for review level and trial level data using RStudio (R version 3.6.1). Because Cochrane requires reviews to be registered with the editorial group to prevent duplicate review topics, analyses were based on an assumption of independence (i.e., no trial was included in more than one review).
      The primary outcome of this study was the proportion of trials included in systematic reviews of interventions with an identifiable clinical trial registration record. Eligible trials for the primary outcome were published in 2005, the first full calendar year that the ICMJE criteria for trial registration came into effect, or later. We also examined potential factors that may be associated with clinical trial registration based on the following criteria: condition (review group), type of intervention (pharmaceutical; medical device; surgical; behavioral, including physiotherapy, diet, and self-care programs; and combined interventions), number of participants (<100; 100 or more), trial date (before 2007; 2007 and after, based on the Food and Drug Administration Amendments Act of 2007 [FDAAA]), and the review authors’ conclusions (favors intervention, favors comparator, inconclusive). Between-group differences were compared using the chi-square test, with P < 0.05 indicating statistical significance.
      Secondary outcomes included the proportion of trials with a clinical trial registration that posted trial results and the proportion of all trials with an open access report. Based on feedback from editorial and peer review, we also analyzed data for trials that were published in the last 10 years (2010–2020) to provide additional insight into more recent trends in clinical trial registration.

      4. Results

      4.1 Review level characteristics

      Among 100 randomly selected reviews from the Cochrane MOSS Network, the majority of reviews evaluated pharmaceutical interventions (56%), performed meta-analysis (93%), and concluded that the test interventions were favorable to comparison interventions (52%) (Table 1). Within specific review groups, these trends were the similar, with the following exceptions: the Back and Neck group evaluated more behavioral interventions (60%; five of which were physiotherapy) than other types; the Oral Health group evaluated more device interventions (60%) than other types, and the review authors’ conclusions were inconclusive in a majority of Oral health and Wounds reviews (60%). There were 2,000 trials included across all reviews (median number of trials included per review was 13, range 5 to 137).
      Table 1Review characteristics overall and by review group (n = 100)
      Review groupIncluded trials: Total number; median (range) per reviewIntervention type: pharmaceutical; device; surgical; behavioral; combination
      Total percentage >100 as reviews may have evaluated more than one type of intervention.
      Meta-analysis: number (%)Review authors' conclusions: Favors intervention; favors comparator; no difference between groups; inconclusive
      Overall (n = 100)2000; 13 (5–137)56; 33; 23; 26; 1293 (93%)52; 3; 6; 39
      Back and neck (n = 10)197; 18 (10–41)2; 0; 2; 6; 010 (100%)6; 0; 1; 3
      Ear, nose, and throat (n = 10)98; 7.5 (5–25)5; 3; 1; 1; 010 (100%)6; 0; 0; 4
      Eyes and vision (n = 30)586; 12.5 (5–137)18; 10; 14; 5; 327 (90%)15; 0; 4; 11
      Musculoskeletal (n = 10)250; 21.5 (7–54)5; 1; 0; 4; 09 (90%)8; 0; 0; 2
      Oral health (n = 10)166; 12.5 (6–32)3; 6; 3; 2; 310 (100%)3; 1; 0; 6
      Pain, palliative and supportive care (n = 10)244; 16.5 (7–62)8; 4; 2; 2; 28 (80%)6; 1; 0; 3
      Skin (n = 10)334; 25 (6–77)8; 5; 0; 6; 39 (90%)5; 0; 1; 4
      Wounds (n = 10)125; 11.5 (7–20)7; 4; 1; 0; 110 (100%)3; 1; 0; 6
      a Total percentage >100 as reviews may have evaluated more than one type of intervention.

      4.2 Trial level characteristics

      The median year of trial publication was 2007, with 823 trials published before 2005, 1,177 trials published in 2005 or after, and 682 trials published in 2010 or after (Table 2). There were 367,137 participants included in 2,000 trials across all reviews (median number of participants per trial was 63, range 1 to 77,015). Most trials used a randomized parallel-group design overall, before and after 2005 (1,704 of 2,000; 85%). Three review groups (musculoskeletal; pain, palliative and supportive care; and skin) included proportionally more trials than three other review groups (ear, nose, and throat; oral health; and wounds). Most trials, especially those published before 2005, were conducted in Europe and North America. In 2005 and after, the proportion of trials conducted in Africa and the Middle East, Asia and the Pacific, South America, and multiple regions increased compared with trials published before 2005. There were more publicly available full-text reports published in 2005 or after (583 of 1,177 trials; 50%) than published before 2005 (241 of 823 trials; 29%). Of 682 trials published in 2010 or after, 352 (52%) had publicly available full-text reports.
      Table 2Characteristics of clinical trials included in systematic reviews
      Trial characteristicsTotal trials (n = 2,000)Trials published before 2005 (n = 823)Trials published in 2005 or after (n = 1,177)Trials published in 2010 or after (n = 682)
      Publication year, median (range)2007

      1958–2018
      1995

      1958–2004
      2010

      2005–2018
      2012

      2010–2018
      Trial participants
       Total367,137181,446230,16190,072
       Median per trial (range)63

      1–77,015
      58

      1–77,015
      68

      4–16,603
      68

      4–4,203
      Trial design, number (percent)
       Parallel-group randomized trial1704 (85%)668 (81%)1036 (88%)608 (89%)
       Cluster randomized trial5 (<1%)1 (<1%)4 (<1%)1 (<1%)
       Cross-over randomized trial105 (5%)74 (9%)31 (3%)19 (3%)
       Within-person randomized trial126 (6%)32 (4%)94 (8%)45 (7%)
       Quasi-randomized trial/unclear60 (3%)48 (6%)12 (1%)9 (1%)
      Clinical topic area, number (percent)
       Back and neck197 (10%)56 (7%)141 (12%)77 (11%)
       Ear, nose, and throat98 (5%)45 (5%)53 (5%)40 (6%)
       Eyes and vision586 (29%)256 (31%)330 (28%)200 (29%)
       Musculoskeletal250 (13%)120 (15%)130 (11%)62 (9%)
       Oral health166 (8%)68 (8%)98 (8%)59 (9%)
       Pain, palliative, supportive care244 (12%)100 (12%)144 (12%)89 (13%)
       Skin334 (17%)125 (15%)209 (18%)117 (17%)
       Wounds125 (6%)53 (6%)72 (6%)38 (6%)
      Geographic region, number (percent)
       Africa/Middle East230 (12%)48 (6%)182 (15%)126 (18%)
       Asia/Pacific435 (22%)93 (11%)342 (29%)216 (32%)
       Europe732 (37%)399 (48%)333 (28%)174 (26%)
       North America430 (22%)237 (29%)193 (16%)102 (15%)
       South America69 (3%)13 (2%)56 (5%)29 (4%)
       Multiple regions95 (5%)26 (3%)69 (6%)35 (5%)
       Not reported9 (<1%)7 (1%)2 (<1%)0

      4.3 Registered vs. nonregistered trials

      We identified a clinical trial registry record for 379 of 1,432 (26%) of trials published since 2000, when ClinicalTrials.gov became publicly available, most of which (97%; 368 of 379) were published since 2005, when the ICMJE criteria for trial registration requirements came into effect. As of 2005, the proportion of trial registration increased to 31% (368 of 1,177 trials), and as of 2010, the proportion of trial registration increased to 38% (261 of 682 trials). Two review groups, musculoskeletal and pain, palliative and supportive care, had proportionally more registered trials than nonregistered trials compared with other review groups (Table 3; Table 4). The majority of registered trials evaluated pharmaceutical interventions (62%); 46% of nonregistered trials evaluated pharmaceutical interventions as of 2005 (44% as of 2010). Registered trials included a median of 120 participants (105,192 overall), compared with a median of 60 (80,499 overall) in nonregistered trials as of 2005 (Table 3). Ninety-three percent of registered trials were published in 2007 or later (median year of publication 2011); 82% of nonregistered trials were published in 2007 or later (median year of publication 2010). Trials from reviews favoring the intervention group were more likely to be registered than nonregistered, whereas trials from reviews with inconclusive results were more likely to be nonregistered than registered as of both 2005 and 2010. Slightly more registered trials had an open access full-text report available (59%) than not available, whereas slightly fewer nonregistered trials had an open access full-text report available (45% as of 2005; 47% as of 2010) than not available.
      Table 3Characteristics of clinical trials published in 2005 or later with vs. without clinical trial registration
      Trial characteristicsTrials with a clinical trial registration (n = 368)Trials without a clinical trial registration (n = 809)
      Clinical topic area, number (percent)
      Chi-square test P < 0.05 comparing registered vs. nonregistered trials.
       Back and neck36 (10%)105 (13%)
       Ear, nose, and throat14 (4%)39 (5%)
       Eyes and vision100 (27%)230 (28%)
       Musculoskeletal56 (15%)74 (9%)
       Oral health15 (4%)83 (10%)
       Pain, palliative and supportive care78 (21%)66 (8%)
       Skin49 (13%)160 (20%)
       Wounds20 (5%)52 (6%)
      Review intervention type, number (percent)
      Total percentage >100 as reviews may have evaluated more than one type of intervention.
       Pharmaceutical229 (62%)370 (46%)
       Device107 (29%)344 (43%)
       Surgical73 (20%)259 (32%)
       Behavioral96 (26%)253 (31%)
       Combination40 (11%)154 (19%)
      Trial participants
      Chi-square test P < 0.05 comparing registered vs. nonregistered trials.
       Total (median per trial)105,192 (120)80,499 (60)
       Less than 100 participants, number (percent)158 (43%)593 (73%)
       100 or more participants, number (percent)210 (57%)216 (27%)
      Date of publication
      Chi-square test P < 0.05 comparing registered vs. nonregistered trials.
       Median (range)2011 (2005–2018)2010 (2005–2018)
       Published before 2007, number (percent)27 (7%)147 (18%)
       Published 2007 or later, number (percent)341 (93%)662 (82%)
      Review authors' conclusions, number (percent)
      Chi-square test P < 0.05 comparing registered vs. nonregistered trials.
       Favors intervention224 (61%)451 (56%)
       Favors comparator15 (4%)11 (1%)
       No difference between groups16 (4%)51 (6%)
       Inconclusive113 (31%)296 (37%)
      Publicly available full-text report (available free of charge), number (percent)
      Chi-square test P < 0.05 comparing registered vs. nonregistered trials.
       Yes216 (59%)367 (45%)
       No152 (41%)442 (55%)
      a Chi-square test P < 0.05 comparing registered vs. nonregistered trials.
      b Total percentage >100 as reviews may have evaluated more than one type of intervention.
      Table 4Characteristics of clinical trials published in 2010 or later with vs. without clinical trial registration
      Trial characteristicsTrials with a clinical trial registration (n = 261)Trials without a clinical trial registration (n = 421)
      Clinical topic area, number (percent)
      Chi-square test P < 0.05 comparing registered vs. nonregistered trials.
       Back and neck26 (10%)51 (12%)
       Ear, nose, and throat12 (5%)28 (7%)
       Eyes and vision78 (30%)122 (29%)
       Musculoskeletal30 (11%)32 (8%)
       Oral health14 (5%)45 (11%)
       Pain, palliative and supportive care55 (21%)34 (8%)
       Skin34 (13%)83 (20%)
       Wounds12 (5%)26 (6%)
      Review intervention type, number (percent)
      Total percentage >100 as reviews may have evaluated more than one type of intervention.
       Pharmaceutical161 (62%)186 (44%)
       Device82 (31%)189 (45%)
       Surgical61 (23%)138 (33%)
       Behavioral66 (25%)126 (30%)
       Combination30 (11%)85 (20%)
      Trial participants
      Chi-square test P < 0.05 comparing registered vs. nonregistered trials.
       Total (median per trial)59,331 (109)30,741 (60)
       Less than 100 participants, number (percent)119 (46%)326 (77%)
       100 or more participants, number (percent)142 (54%)95 (23%)
      Date of publication
       Median (range)2013 (2010–2018)2012 (2010–2018)
      Review authors' conclusions, number (percent)
      Chi-square test P < 0.05 comparing registered vs. nonregistered trials.
       Favors intervention155 (59%)224 (53%)
       Favors comparator10 (4%)3 (1%)
       No difference between groups10 (4%)14 (3%)
       Inconclusive86 (33%)180 (43%)
      Publicly available full-text report (available free of charge), number (percent)
      Chi-square test P < 0.05 comparing registered vs. nonregistered trials.
       Yes155 (59%)197 (47%)
       No106 (41%)224 (53%)
      a Chi-square test P < 0.05 comparing registered vs. nonregistered trials.
      b Total percentage >100 as reviews may have evaluated more than one type of intervention.
      As of 2005, about one-third of registered trials (114 of 368; 31%) provided results for at least one outcome within the registry record. Most trials were retrospectively registered (233 of 368; 63%); 135 of 368 (37%) were registered prospectively before the enrollment of the first participant. As of 2010, one-third of registered trials (87 of 261; 33%) provided results for at least one outcome within the registry record. Still, most trials were retrospectively registered (151 of 261; 58%), but some improvement was seen with 110 of 261 (42%) trials registered prospectively before the enrollment of the first participant.

      4.4 Trial registration by year of publication

      Overall, there is an increasing trend in the number of registered trials since 2005 and 2010, although for only 2 years, 2015 (n = 65) and 2018 (n = 4), was the percent of registration more than 50% of trials published in those years (Fig. 1). The cumulative percentage of trials registered increased between 2005 and 2015 but remains less than one-third of all trials identified in this project (Fig. 2). There were slight increases in the number of trial registrations with results posted from 2008 to 2013; however, the cumulative percentage of registered trials included in recently published systematic reviews still remains very low at less than 10%.
      Figure thumbnail gr1
      Fig. 1Percent of clinical trials with clinical trial registration and with results posted by year of publication.
      Figure thumbnail gr2
      Fig. 2Cumulative percent of clinical trials with clinical trial registration and with results posted by year of publication.

      5. Discussion

      Although we observed some improvement in the registration of clinical trials since 2000, especially after 2005 when trial registration was endorsed by the ICMJE (change from 25% to 31%), there does not seem to be strong consistency of trial registration in practice within the topic areas evaluated in our sample of systematic reviews. Fewer than one-third of trials identified in this study had an accessible clinical trial registration record, even when conducting in-depth searches of multiple sources and allowing retrospective registration. Even fewer, less than 10%, provided efficacy or safety results as part of the trial registration record. These deficiencies in clinical trial registration and reporting negatively affect the confidence and reliability of the evidence ecosystem which they underpin.
      Previous studies have reported poor compliance with clinical trial registration requirements in both ClinicalTrials.gov [
      • Anderson M.L.
      • Chiswell K.
      • Peterson E.D.
      • Tasneem A.
      • Topping J.
      • Califf R.M.
      Compliance with results reporting at ClinicalTrials.gov.
      ,
      • DeVito N.
      • Bacon S.
      • Goldacre B.
      Compliance with legal requirement to report clinical trial results on ClinicalTrials.gov: a cohort study.
      ,
      • Créquit P.
      • Boutron I.
      • Meerpohl J.
      • Williams H.C.
      • Craig J.
      • Ravaud P.
      Future of evidence ecosystem series: 2. current opportunities and need for better tools and methods.
      ] and the EU Clinical Trials Register [
      • Créquit P.
      • Boutron I.
      • Meerpohl J.
      • Williams H.C.
      • Craig J.
      • Ravaud P.
      Future of evidence ecosystem series: 2. current opportunities and need for better tools and methods.
      ,
      • Denneny C.
      • Bourne S.
      • Kolstoe S.E.
      Registration audit of clinical trials given a favourable opinion by UK research ethics committees.
      ,
      • Goldacre B.
      • Devito N.J.
      • Heneghan C.
      • Irving F.
      • Bacon S.
      • Fleminger J.
      • et al.
      Compliance with requirement to report results on the EU Clinical Trials Register: cohort study and web resource.
      ] in the range of 39% to 50%. The lower proportion found in this study (31% as of 2005) is likely due to the time needed to conduct a systematic review after the included trials have been completed and lags in publication. Even so, by reviewing the status of trial registration without time restrictions, our results represent an overestimate of the proportion of trials included in systematic reviews that adhered to trial registration guidelines. Although trials should be registered before enrollment of the first participant, most of the trials identified in our study were registered retrospectively (63%). Furthermore, trial results for interventions requiring regulatory approval should be made available within 12 months of the study conclusion. We accepted any posted trial result regardless of when the results were posted and still found only 10% with trial results. Finally, we searched for trial registration records more recently (as of March 1, 2020) than when the original systematic review authors conducted searches for their reviews, all of which were published between 2014 and 2019.
      Of all registered trials, 93% were published in 2007, the year FDAAA was enacted, or later. We also found pharmaceutical interventions made up the most common type of intervention among registered trials included in systematic reviews (62%). These findings are consistent with other studies that suggest better compliance with both ICMJE and FDAAA trial registration requirements for trials conducted for the specific purpose of new drug approvals, with up to 100% compliance for specific drugs [
      • Phillips A.T.
      • Desai N.R.
      • Krumholz H.M.
      • Zou C.X.
      • Miller J.E.
      • Ross J.S.
      Association of the FDA Amendment Act with trial registration, publication, and outcome reporting.
      ,
      • Lassman S.M.
      • Shopshear O.M.
      • Jazic I.
      • Ulrich J.
      • Francer J.
      Clinical trial transparency: a reassessment of industry compliance with clinical trial registration and reporting requirements in the United States.
      ]. However, although also regulated by FDAAA, only 24% of device trials in our study were registered as of 2005 and 30% as of 2010.
      The Final Rule of FDAAA, developed in 2016, set out to clarify which trials are required to comply with federal trial registration and reporting regulations in the United States [
      • Zarin D.A.
      • Tse T.
      • Williams R.J.
      • Carr S.
      Trial reporting in ClinicalTrials.gov - the final rule.
      ]. Based on a study by DeVito et al., complete compliance plateaued from July 2018 to September 2019 at around 40% [
      • DeVito N.
      • Bacon S.
      • Goldacre B.
      Compliance with legal requirement to report clinical trial results on ClinicalTrials.gov: a cohort study.
      ], suggesting that, even in the era of the Final Rule, reporting of clinical trial results still falls short. In the context of the evidence ecosystem, more time is required to fully assess the impact of the Final Rule on clinical trial compliance and the reliability of systematic reviews. What also remains unclear is how to address the lack of oversight of trial registration and reporting for trials that influence patient care, but not covered by FDAAA and other regulatory agencies, especially with respect to surgical and behavioral (including physiotherapy) trials.
      The scientific community at large also has an important role in better enforcing clinical trial registration. A study by Cook et al. [
      • Cook C.
      • Checketts J.X.
      • Atakpo P.
      • Nelson N.
      • Vassar M.
      How well are reporting guidelines and trial registration used by dermatology journals to limit bias? A meta-epidemiological study.
      ] found that dermatology journals that required or recommended trial registration when considering articles for publication had higher rates of trial registration reporting (72%) than those without formal trial registration policies (38%). Similar studies in other disease areas also have shown increased reporting of trial registration among journals with policies that require or recommend trial registration compared with those that do not [
      • Checketts J.X.
      • Sims M.T.
      • Detweiler B.
      • Middlemist K.
      • Jones J.
      • Vassar M.
      An evaluation of reporting guidelines and clinical trial registry requirements among Orthopaedic Surgery Journals.
      ,
      • Checketts J.X.
      • Cook C.
      • Imani S.
      • Duckett L.
      • Vassar M.
      An evaluation of reporting guidelines and clinical trial registry requirements among plastic surgery journals.
      ,
      • Sims M.T.
      • Bowers A.M.
      • Fernan J.M.
      • Dormire K.D.
      • Herrington J.M.
      • Vassar M.
      Trial registration and adherence to reporting guidelines in cardiovascular journals.
      ]. These differences in reporting provide evidence that, by imposing policies at the level of journal publication, the percentage of trials published with registration information can be improved. Professional societies can also adopt the CONSORT Statement extension for abstracts [
      • Hopewell S.
      • Clarke M.
      • Moher D.
      • et al.
      CONSORT for reporting randomized controlled trials in journal and conference abstracts: explanation and elaboration.
      ] and require clinical trial registration information to be reported as part of the abstract submission and acceptance process for conferences. Likewise, internal review boards and ethic committees could require trial registration before approving the start of patient enrollment. Even though it is not a formal part of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses checklist [
      • Moher D.
      • Liberati A.
      • Tetzlaff J.
      • Altman D.G.
      PRISMA Group
      Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement.
      ], systematic reviewers often document clinical trial registration of studies included in their reviews and use the information provided in the trial registration to assess for selective outcome reporting and other sources of bias. Although not ideal, an approach of implementing clinical trial registration requirements at various stages of the evidence ecosystem seems the only feasible way to ensure that these standards will be met as no one method seems capable of ensuring clinical trial registration for all trials.

      6. Conclusions

      In this study, we systematically examined the proportion of clinical trial registration among trials included in recently published systematic reviews of interventions. Although some improvement in clinical trial registration has been observed in recent years, the proportion of registered clinical trials included in recently published systematic reviews remains less than desirable. Systematic reviews, to provide the best level of evidence for decision makers, should be based on properly conducted and completely reported clinical trials. Access to unbiased and complete trial information needed to adequately judge the quality and strength of evidence plays a critical role in the evidence-based health care ecosystem and trustworthiness of medical research.

      References

        • Guyatt G.H.
        • Sackett D.L.
        • Sinclair J.C.
        • Hayward R.
        • Cook D.J.
        • Cook R.J.
        Users' guides to the medical literature. IX. A method for grading health care recommendations. Evidence-Based Medicine Working Group.
        JAMA. 1995; 274: 1800-1804
        • De Angelis C.
        • Drazen J.M.
        • Frizelle F.A.
        • Haug C.
        • Hoey J.
        • Horton R.
        • et al.
        Clinical trial registration: a statement from the international committee of medical journal Editors.
        Ann Intern Med. 2004; 141: 477-478
      1. Food and Drug Administration Amendments Act of 2007.
        (Available at)
        • European Medicines Agency
        Posting of clinical trial summary results in European Clinical Trials Database (EudraCT) to become mandatory for sponsors as of 21 July 2014.
        2014 (Available at)
        • AllTrials
        About AllTrials. All Trials Registered | All Results Reported.
        https://www.alltrials.net/find-out-more/about-alltrials/
        Date: 2013
        Date accessed: April 6, 2020
        • Anderson M.L.
        • Chiswell K.
        • Peterson E.D.
        • Tasneem A.
        • Topping J.
        • Califf R.M.
        Compliance with results reporting at ClinicalTrials.gov.
        N Engl J Med. 2015; 372: 1031-1039
        • DeVito N.
        • Bacon S.
        • Goldacre B.
        Compliance with legal requirement to report clinical trial results on ClinicalTrials.gov: a cohort study.
        Lancet. 2020; 395: 361-369
        • Créquit P.
        • Boutron I.
        • Meerpohl J.
        • Williams H.C.
        • Craig J.
        • Ravaud P.
        Future of evidence ecosystem series: 2. current opportunities and need for better tools and methods.
        J Clin Epidemiol. 2020; 123: 143-152
        • Denneny C.
        • Bourne S.
        • Kolstoe S.E.
        Registration audit of clinical trials given a favourable opinion by UK research ethics committees.
        BMJ Open. 2019; 9: e026840
        • Goldacre B.
        • Devito N.J.
        • Heneghan C.
        • Irving F.
        • Bacon S.
        • Fleminger J.
        • et al.
        Compliance with requirement to report results on the EU Clinical Trials Register: cohort study and web resource.
        BMJ. 2018; 362 (Article Number k3218)
        • Phillips A.T.
        • Desai N.R.
        • Krumholz H.M.
        • Zou C.X.
        • Miller J.E.
        • Ross J.S.
        Association of the FDA Amendment Act with trial registration, publication, and outcome reporting.
        Trials. 2017; 18: 333
        • Lassman S.M.
        • Shopshear O.M.
        • Jazic I.
        • Ulrich J.
        • Francer J.
        Clinical trial transparency: a reassessment of industry compliance with clinical trial registration and reporting requirements in the United States.
        BMJ Open. 2017; 7: e015110
        • Zarin D.A.
        • Tse T.
        • Williams R.J.
        • Carr S.
        Trial reporting in ClinicalTrials.gov - the final rule.
        N Engl J Med. 2016; 375: 1998-2004
        • Cook C.
        • Checketts J.X.
        • Atakpo P.
        • Nelson N.
        • Vassar M.
        How well are reporting guidelines and trial registration used by dermatology journals to limit bias? A meta-epidemiological study.
        Br J Dermatol. 2018; 178: 1433-1434
        • Checketts J.X.
        • Sims M.T.
        • Detweiler B.
        • Middlemist K.
        • Jones J.
        • Vassar M.
        An evaluation of reporting guidelines and clinical trial registry requirements among Orthopaedic Surgery Journals.
        J Bone Joint Surg Am. 2018; 100: e15
        • Checketts J.X.
        • Cook C.
        • Imani S.
        • Duckett L.
        • Vassar M.
        An evaluation of reporting guidelines and clinical trial registry requirements among plastic surgery journals.
        Ann Plast Surg. 2018; 81: 215-219
        • Sims M.T.
        • Bowers A.M.
        • Fernan J.M.
        • Dormire K.D.
        • Herrington J.M.
        • Vassar M.
        Trial registration and adherence to reporting guidelines in cardiovascular journals.
        Heart. 2018; 104: 753-759
        • Hopewell S.
        • Clarke M.
        • Moher D.
        • et al.
        CONSORT for reporting randomized controlled trials in journal and conference abstracts: explanation and elaboration.
        PLoS Med. 2008; 5: e20
        • Moher D.
        • Liberati A.
        • Tetzlaff J.
        • Altman D.G.
        • PRISMA Group
        Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement.
        PLoS Med. 2009; 6: e1000097