Advertisement

A meta-research study of randomized controlled trials found infrequent and delayed availability of protocols

  • Author Footnotes
    1 Contributed equally (shared first authorship).
    Christof Manuel Schönenberger
    Correspondence
    Corresponding author. Meta-Research Centre, Department of Clinical Research, University of Basel and University Hospital Basel, Spitalstrasse 12, 4056 Basel, Switzerland. Tel.: +41-78-929-9881; fax: +41-61-265-3109.
    Footnotes
    1 Contributed equally (shared first authorship).
    Affiliations
    Meta-Research Centre, Department of Clinical Research, University of Basel and University Hospital Basel, Basel, Switzerland
    Search for articles by this author
  • Author Footnotes
    1 Contributed equally (shared first authorship).
    Alexandra Griessbach
    Footnotes
    1 Contributed equally (shared first authorship).
    Affiliations
    Meta-Research Centre, Department of Clinical Research, University of Basel and University Hospital Basel, Basel, Switzerland
    Search for articles by this author
  • Ala Taji Heravi
    Affiliations
    Meta-Research Centre, Department of Clinical Research, University of Basel and University Hospital Basel, Basel, Switzerland
    Search for articles by this author
  • Dmitry Gryaznov
    Affiliations
    Meta-Research Centre, Department of Clinical Research, University of Basel and University Hospital Basel, Basel, Switzerland
    Search for articles by this author
  • Viktoria L. Gloy
    Affiliations
    Meta-Research Centre, Department of Clinical Research, University of Basel and University Hospital Basel, Basel, Switzerland
    Search for articles by this author
  • Szimonetta Lohner
    Affiliations
    Cochrane Hungary, Clinical Centre of the University of Pécs, Medical School, University of Pécs, Pécs, Hungary

    Department of Public Health Medicine, Medical School, University of Pécs, Pécs, Hungary
    Search for articles by this author
  • Katharina Klatte
    Affiliations
    Meta-Research Centre, Department of Clinical Research, University of Basel and University Hospital Basel, Basel, Switzerland
    Search for articles by this author
  • Nilabh Ghosh
    Affiliations
    Meta-Research Centre, Department of Clinical Research, University of Basel and University Hospital Basel, Basel, Switzerland
    Search for articles by this author
  • Hopin Lee
    Affiliations
    Oxford Clinical Trials Research Unit/Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford, UK
    Search for articles by this author
  • Anita Mansouri
    Affiliations
    Oxford Clinical Trials Research Unit/Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford, UK
    Search for articles by this author
  • Ioana R. Marian
    Affiliations
    Oxford Clinical Trials Research Unit/Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford, UK
    Search for articles by this author
  • Ramon Saccilotto
    Affiliations
    Clinical Trial Unit, Department of Clinical Research, University of Basel and University Hospital Basel, Basel, Switzerland
    Search for articles by this author
  • Edris Nury
    Affiliations
    Institute for Evidence in Medicine (for Cochrane Germany Foundation), Faculty of Medicine and Medical Center, University of Freiburg, Freiburg, Germany

    Department of General Practice and Primary Care, Medical Center Hamburg-Eppendorf - UKE, Hamburg, Germany
    Search for articles by this author
  • Jason W. Busse
    Affiliations
    Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, Canada

    Department of Anesthesia, McMaster University, Hamilton, Canada
    Search for articles by this author
  • Belinda von Niederhäusern
    Affiliations
    Roche Pharma AG, Grenzach-Wyhlen, Germany
    Search for articles by this author
  • Dominik Mertz
    Affiliations
    Department of Medicine, McMaster University, Hamilton, Canada
    Search for articles by this author
  • Anette Blümle
    Affiliations
    Clinical Trials Unit, Faculty of Medicine and Medical Center, University of Freiburg, Freiburg, Germany
    Search for articles by this author
  • Ayodele Odutayo
    Affiliations
    Oxford Clinical Trials Research Unit/Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford, UK

    Department of Anesthesia, McMaster University, Hamilton, Canada
    Search for articles by this author
  • Sally Hopewell
    Affiliations
    Oxford Clinical Trials Research Unit/Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford, UK
    Search for articles by this author
  • Author Footnotes
    2 Contributed equally (shared last authorship).
    Benjamin Speich
    Footnotes
    2 Contributed equally (shared last authorship).
    Affiliations
    Meta-Research Centre, Department of Clinical Research, University of Basel and University Hospital Basel, Basel, Switzerland

    Oxford Clinical Trials Research Unit/Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford, UK
    Search for articles by this author
  • Author Footnotes
    2 Contributed equally (shared last authorship).
    Matthias Briel
    Footnotes
    2 Contributed equally (shared last authorship).
    Affiliations
    Meta-Research Centre, Department of Clinical Research, University of Basel and University Hospital Basel, Basel, Switzerland

    Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, Canada
    Search for articles by this author
  • Author Footnotes
    1 Contributed equally (shared first authorship).
    2 Contributed equally (shared last authorship).
Open AccessPublished:May 29, 2022DOI:https://doi.org/10.1016/j.jclinepi.2022.05.014

      Abstract

      Objectives

      Availability of randomized controlled trial (RCT) protocols is essential for the interpretation of trial results and research transparency.

      Study Design and Setting

      In this study, we determined the availability of RCT protocols approved in Switzerland, Canada, Germany, and the United Kingdom in 2012. For these RCTs, we searched PubMed, Google Scholar, Scopus, and trial registries for publicly available protocols and corresponding full-text publications of results. We determined the proportion of RCTs with (1) publicly available protocols, (2) publications citing the protocol, and (3) registries providing a link to the protocol. A multivariable logistic regression model explored factors associated with protocol availability.

      Results

      Three hundred twenty-six RCTs were included, of which 118 (36.2%) made their protocol publicly available; 56 (47.6% 56 of 118) provided as a peer-reviewed publication and 48 (40.7%, 48 of 118) provided as supplementary material. A total of 90.9% (100 of 110) of the protocols were cited in the main publication, and 55.9% (66 of 118) were linked in the clinical trial registry. Larger sample size (>500; odds ratio [OR] = 5.90, 95% confidence interval [CI], 2.75–13.31) and investigator sponsorship (OR = 1.99, 95% CI, 1.11–3.59) were associated with increased protocol availability. Most protocols were made available shortly before the publication of the main results.

      Conclusion

      RCT protocols should be made available at an early stage of the trial.

      Keywords

      What is new?

        Key findings

      • One-third of RCTs provided a publicly available protocol; greater availability was associated with investigator sponsorship and larger sample size.
      • Protocols were typically made available shortly before the publication of RCT results.
      • Protocols of investigator-sponsored trials were made available earlier than those of industry-sponsored trials.
      • Only about half of the clinical trials registry entries provided a link to the corresponding protocol.

        What this adds to what was known?

      • Overall poor availability of protocols in a representative sample.
      • Multivariable regression model analysing factors associated with increased protocol availability.

        What is the implication and what should change now?

      • RCT protocols should be made available through clinical trial registries and mandatory requirements by journals.

      1. Introduction

      Randomized controlled trials (RCTs) are important for clinical decision-making [
      • Zabor E.C.
      • Kaizer A.M.
      • Hobbs B.P.
      Randomized controlled trials.
      ], and publicly available protocols help ensure consistency of trial processes, ethical conduct, transparency, and valid research results [
      • Chan A.W.
      • Tetzlaff J.M.
      • Gotzsche P.C.
      • Altman D.G.
      • Mann H.
      • Berlin J.A.
      • et al.
      SPIRIT 2013 explanation and elaboration: guidance for protocols of clinical trials.
      ,
      • Idänpään-heikkilä J.E.
      WHO guidelines for good clinical practice (GCP) for trials on pharmaceutical products: responsibilities of the investigator.
      ]. The International Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use Good Clinical Practice guidelines require every protocol to be reviewed and approved by an ethics committee before enrollment of the first patient [
      • Vijayananthan A.
      • Nawawi O.
      The importance of good clinical practice guidelines and its role in clinical trials.
      ], and making protocols publicly available has been promoted as good research practice over the last decade [
      • Li T.
      • Boutron I.
      • Salman R.A.S.
      • Cobo E.
      • Flemyng E.
      • Grimshaw J.M.
      • et al.
      Review and publication of protocol submissions to trials – what have we learned in 10 years?.
      ].
      Previous research has investigated the importance of making protocols available to the public [
      • Sender D.
      • Clark J.
      • Hoffmann T.C.
      Analysis of articles directly related to randomized trials finds poor protocol availability and inconsistent linking of articles.
      ,
      • Hoffmann T.
      • Glasziou P.
      • Beller E.
      • Goldacre B.
      • Chalmers I.
      Focus on sharing individual patient data distracts from other ways of improving trial transparency.
      ,
      • Chan A.W.
      • Hróbjartsson A.
      Promoting public access to clinical trial protocols: challenges and recommendations.
      ,
      • Altman D.G.
      • Furberg C.D.
      • Grimshaw J.M.
      • Rothwell P.M.
      Lead editorial: trials – using the opportunities of electronic publishing to improve the reporting of randomised trials.
      ]. Protocol availability increases research quality on account of detailed consideration of methodological procedures [
      • Altman D.G.
      • Furberg C.D.
      • Grimshaw J.M.
      • Shanahan D.R.
      Linked publications from a single trial: a thread of evidence.
      ]. Moreover, protocols contain additional information which aids with the interpretation of study results and reduces bias, by predetermining outcomes and reducing selective outcome reporting (“cherry-picking”) and outcome switching [
      • Chan A.W.
      • Tetzlaff J.M.
      • Gotzsche P.C.
      • Altman D.G.
      • Mann H.
      • Berlin J.A.
      • et al.
      SPIRIT 2013 explanation and elaboration: guidance for protocols of clinical trials.
      ,
      • Chan A.W.
      • Hróbjartsson A.
      Promoting public access to clinical trial protocols: challenges and recommendations.
      ,
      • Altman D.G.
      • Furberg C.D.
      • Grimshaw J.M.
      • Rothwell P.M.
      Lead editorial: trials – using the opportunities of electronic publishing to improve the reporting of randomised trials.
      ,
      • Chan A.W.
      • Hrobjartsson A.
      • Jorgensen K.J.
      • Gotzsche P.C.
      • Altman D.G.
      Discrepancies in sample size calculations and data analyses reported in randomised trials: comparison of publications with protocols.
      ,
      • Mayo-Wilson E.
      • Li T.
      • Fusco N.
      • Bertizzolo L.
      • Canner J.K.
      • Cowley T.
      • et al.
      Cherry-picking by trialists and meta-analysts can drive conclusions about intervention efficacy.
      ,
      • Chan A.W.
      • Pello A.
      • Kitchen J.
      • Axentiev A.
      • Virtanen J.I.
      • Liu A.
      • et al.
      Association of trial registration with reporting of primary outcomes in protocols and publications.
      ]. Publishing and making protocols publicly available may reduce misreporting of results, improve the design of future trials, increase transparency, and promote ethical compliance [
      • Chan A.W.
      • Hrobjartsson A.
      • Jorgensen K.J.
      • Gotzsche P.C.
      • Altman D.G.
      Discrepancies in sample size calculations and data analyses reported in randomised trials: comparison of publications with protocols.
      ,
      • Mathieu S.
      Comparison of registered and published primary outcomes in randomized controlled trials.
      ,
      • Dwan K.
      • Altman D.G.
      • Clarke M.
      • Gamble C.
      • Higgins J.P.T.
      • Sterne J.A.C.
      • et al.
      Evidence for the selective reporting of analyses and discrepancies in clinical trials: a systematic review of cohort studies of clinical trials.
      ,
      • Chalmers I.
      • Glasziou P.
      Avoidable waste in the production and reporting of research evidence.
      ,
      • Moher D.
      • Liberati A.
      • Tetzlaff J.
      • Altman D.G.
      The PRISMA Group
      Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement.
      ,
      • Shamseer L.
      • Moher D.
      • Clarke M.
      • Ghersi D.
      • Liberati A.
      • Petticrew M.
      • et al.
      Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015: elaboration and explanation.
      ]. These benefits are even more substantial if protocol availability precedes the reporting of trial results. Timely availability of protocols informs researchers about the original intent of the study and if this intent was maintained during conduct [
      • Nosek B.A.
      • Ebersole C.R.
      • DeHaven A.C.
      • Mellor D.T.
      The preregistration revolution.
      ]. This is especially important since changes in outcomes and conflation between exploratory vs. confirmatory purposes (hypothesis generation and hypothesis testing) are common [
      • Nosek B.A.
      • Ebersole C.R.
      • DeHaven A.C.
      • Mellor D.T.
      The preregistration revolution.
      ,
      • Hardwicke T.E.
      • Ioannidis J.P.A.
      Mapping the universe of registered reports.
      ], lead to bias [
      • Nosek B.A.
      • Ebersole C.R.
      • DeHaven A.C.
      • Mellor D.T.
      The preregistration revolution.
      ,
      • Ioannidis J.P.A.
      Why most published research findings are false.
      ], and are known to inflate the type I error rate and contribute to the replication crisis [
      • Nosek B.A.
      • Ebersole C.R.
      • DeHaven A.C.
      • Mellor D.T.
      The preregistration revolution.
      ]. Later versions of protocols may not reflect the original purpose of the study. Finally, timely availability of protocols provides investigators with greater confidence in their statistical inference [
      • Chan A.W.
      • Pello A.
      • Kitchen J.
      • Axentiev A.
      • Virtanen J.I.
      • Liu A.
      • et al.
      Association of trial registration with reporting of primary outcomes in protocols and publications.
      ,
      • Nosek B.A.
      • Ebersole C.R.
      • DeHaven A.C.
      • Mellor D.T.
      The preregistration revolution.
      ,
      • Uppstad P.H.
      A rationale for publishing peer-reviewed study protocols in the Nordic Journal of Literacy Research to increase scientific rigour.
      ,
      • Errington T.M.
      • Denis A.
      • Perfito N.
      • Iorns E.
      • Nosek B.A.
      Challenges for assessing replicability in preclinical cancer biology.
      ].
      Although the benefits of making protocols publicly available are well established, data quantifying their availability and the timing of their availability are limited to three studies [
      • Sender D.
      • Clark J.
      • Hoffmann T.C.
      Analysis of articles directly related to randomized trials finds poor protocol availability and inconsistent linking of articles.
      ,
      • Lucey M.
      • Clark J.
      • Glasziou P.
      Public availability of trial protocols.
      ,
      • Spence O.
      • Hong K.
      • Onwuchekwa Uba R.
      • Doshi P.
      Availability of study protocols for randomized trials published in high-impact medical journals: a cross-sectional analysis.
      ]. Sender et al. [
      • Sender D.
      • Clark J.
      • Hoffmann T.C.
      Analysis of articles directly related to randomized trials finds poor protocol availability and inconsistent linking of articles.
      ] primarily focused on nonpharmacologic trials, Lucey et al. [
      • Lucey M.
      • Clark J.
      • Glasziou P.
      Public availability of trial protocols.
      ] only included RCTs that were submitted to the Lancet, and Spence et al. [
      • Spence O.
      • Hong K.
      • Onwuchekwa Uba R.
      • Doshi P.
      Availability of study protocols for randomized trials published in high-impact medical journals: a cross-sectional analysis.
      ] only included RCTs whose results were only published in high-impact journals [
      • Spence O.
      • Hong K.
      • Onwuchekwa Uba R.
      • Doshi P.
      Availability of study protocols for randomized trials published in high-impact medical journals: a cross-sectional analysis.
      ]. Overall, there is limited generalizability concerning these findings [
      • Sender D.
      • Clark J.
      • Hoffmann T.C.
      Analysis of articles directly related to randomized trials finds poor protocol availability and inconsistent linking of articles.
      ,
      • Lucey M.
      • Clark J.
      • Glasziou P.
      Public availability of trial protocols.
      ].
      In this study, we aimed to determine the proportion of publicly available protocols, as well as the timing of their availability, from a random sample of RCTs approved by ethics committees in Switzerland, Germany, Canada, and the United Kingdom in 2012 [
      • Gryaznov D.
      • Odutayo A.
      • von Niederhäusern B.
      • Speich B.
      • Kasenda B.
      • Ojeda-Ruiz E.
      • et al.
      Rationale and design of repeated cross-sectional studies to evaluate the reporting quality of trial protocols: the adherence to SPIrit REcommendations (ASPIRE) study and associated projects.
      ]. Moreover, we determined the public source of the protocols, if they were cited by the corresponding main publication, and if they were referenced in a clinical trial registry. Finally, we investigated factors associated with increased protocol availability in a multivariable logistic regression model.

      2. Methods

      Our study sample was derived from a previous study (Adherence to Spirit Recommendations Study [ASPIRE]) and included a random convenience sample of RCTs approved by ethics committees in Switzerland (Basel, Bellinzona, Bern, Geneva, Lausanne, St. Gallen, and Thurgau), Canada (Hamilton), the United Kingdom (the Bristol office of the UK National Research Ethics Service responsible for 19 research ethics committees in the United Kingdom), and Germany (Freiburg) in 2012 (see supplementary material for more details) [
      • Gryaznov D.
      • Odutayo A.
      • von Niederhäusern B.
      • Speich B.
      • Kasenda B.
      • Ojeda-Ruiz E.
      • et al.
      Rationale and design of repeated cross-sectional studies to evaluate the reporting quality of trial protocols: the adherence to SPIrit REcommendations (ASPIRE) study and associated projects.
      ]. In a substudy of the ASPIRE project (DISCOntinued trials II [DISCO II]), we assessed the proportion of RCTs that were nonregistered, discontinued, and unpublished 10 years after ethical approval [
      • Speich B.
      • Gryaznov D.
      • Busse J.W.
      • Gloy V.L.
      • Lohner S.
      • Klatte K.
      • et al.
      Nonregistration, discontinuation, and nonpublication of randomized trials: a repeated metaresearch analysis.
      ]. For DISCO II, we excluded RCTs if they were still ongoing, were never started (information collected from ethics committees or from investigators), or were identified as pilot or feasibility studies (see the flow diagram in supplementary material). Building on this sample, we assessed the proportion of study protocols that were made publicly available and at what time point over the course of the trial.

      2.1 Data collection

      Baseline characteristics such as trial design, sponsorship, intervention type, country, multicenter or single-center status, and planned sample size were extracted by the ASPIRE research team for each RCT from study protocols approved by research ethics boards [
      • Gryaznov D.
      • Odutayo A.
      • von Niederhäusern B.
      • Speich B.
      • Kasenda B.
      • Ojeda-Ruiz E.
      • et al.
      Rationale and design of repeated cross-sectional studies to evaluate the reporting quality of trial protocols: the adherence to SPIrit REcommendations (ASPIRE) study and associated projects.
      ].
      The search for trial registration and publications of results is described in detail in the DISCO II study [
      • Speich B.
      • Gryaznov D.
      • Busse J.W.
      • Gloy V.L.
      • Lohner S.
      • Klatte K.
      • et al.
      Nonregistration, discontinuation, and nonpublication of randomized trials: a repeated metaresearch analysis.
      ]. In brief, we searched the World Health Organization International Clinical Trials Registry Platform, ClinicalTrials.gov, the European Union Clinical Trial Registry, International Standard Randomised Controlled Trial Number registry, and the Google search engine to identify trial registration details. We searched PubMed, Google Scholar, and Scopus for publication of trial results. Both searches were conducted in duplicate and included searching for (i) full titles, (ii) short titles, (iii) study acronyms, and (iv) the study population and intervention (with or without adding the control group).
      The corresponding study protocols and publications of primary results were identified through screening of trial registries, PubMed, Google Scholar, and Scopus using the same search strategy. We also reviewed trial registries to identify protocols that were linked or made available as a separate file. For all trial publications retrieved, we searched for the corresponding protocol by screening the citations and supplementary material. A protocol was defined as a document containing the essential items of the Standard Protocol Items: Recommendations for Interventional Trials guideline; however, we did not examine the completeness or quality protocols. All searches and data extractions were conducted independently and in duplicate by pairs of trained reviewers. Disagreements between investigators were resolved by discussion to achieve consensus. All searches were conducted up to February 2022.
      For each RCT, we extracted the start date of the trial from the registry, which was determined as the date of first patient enrollment, the format of the available protocols (i.e., as peer-reviewed publication, supplementary material to the primary result publication, Portable Document Format available in the trial registry, or other), and the date of publication for both the protocol and publication of primary results. We assumed the availability date was equal to that of the published primary results when protocols were made available as supplementary material, expect when previously published protocols were identified. If a protocol was available as both a peer-reviewed publication and supplementary material, we coded the format as “peer-reviewed publication.” Finally, we documented whether available protocols were cited in the publication of primary results or found (linked) in a clinical trial registry.

      2.2 Analysis

      We summarized characteristics of included RCTs using the median and interquartile range (IQR) for continuous variables and numbers accompanied by percentages for categorical variables. We produced a multivariable logistic regression model (including calculation of odds ratio, 95% confidence intervals, and P values), in which the dependent variable was protocol availability, and the independent variables were sample size (<100, 100–500, >500), sponsorship (industry vs. investigator), multicenter vs. single-center trials, and drug vs. nondrug interventions.
      We determined the time point the protocol was made available during the trial by calculating the relative time ratio (RTR) as follows:
      RTR=numberofdaysfromstartoftrialtoprotocolpublicationnumberofdaysfromthestartoftrialtopublicationofprimaryresults


      The start of the trial was defined as the date of first patient enrollment, which was extracted from the registry entry. The dates of availability for protocols and publications were extracted from the journal in the case of published protocols and results (date available online) and from the clinical trial registry or website for non–peer-reviewed sources. An RTR of <0 indicates the protocol was published before the start of the trial, whereas an RTR >1 indicates the protocol was made available after the publication of the primary results. We conducted a sensitivity analysis for the RTR excluding protocols that were only available as supplementary material (see appendix). A P value of 0.05 was considered statistically significant for all analyses. We used R, version 1.4.1103, for all data management and analyses.

      3. Results

      We included 326 RCTs in this study (see the flow diagram in supplementary material). The median number of participants was 262 (IQR = 100–600).
      Approximately half of the trials were industry initiated (179 of 326; 54.9%), the majority were multicenter studies (266 of 326; 81.6%), and most used a parallel group study design (296 of 326; 90.8%) (Table 1).
      Table 1Characteristics of included RCTs and protocol availability
      Trial characteristicsAll included RCTsRCTs with a publicly available protocol
      All ethically approved RCTs from our sample326 (100%)118/326 (36.2%)
      Sponsorship
       Investigator -sponsored147 (45.1%)56/147 (38.1%)
       Industry -sponsored179 (54.9%)62/179 (34.6%)
      Study designs
       Parallel -arm296 (90.8%)109/296 (36.8%)
       Factorial10 (3.1%)6/10 (60.0%)
       Cluster4 (1.2%)3/4 (75.0%)
       Others
      Others: crossover, split body, and unsure.
      16 (4.9%)0/16 (0%)
      Drug vs. non-drug
       Drug207 (63.5%)77/207 (37.2%)
       Non-drug119 (36.5%)41/119 (34.5%)
      Single-center or multicenter
       Single-center60 (18.4%)12/60 (20.0%)
       Multicenter266 (81.6%)106/266 (39.8%)
      National67 (25.2%)27/67 (40.3%)
      International199 (74.8%)79/199 (39.6%)
      Number of participants
       <10073 (22.4%)14/73 (19.2%)
       100–500151 (46.3%)45/151 (29.8%)
       >500102 (31.3%)59/102 (57.8%)
      Country of ethical approval
       Switzerland165 (50.6%)54/165 (32.7%)
       United Kingdom89 (27.3%)35/89 (39.3%)
       Germany37 (11.3%)9/37 (24.3%)
       Canada35 (10.7%)20/35 (57.1%)
      Registration
       Registered306 (93.9%)118/306 (38.6%)
       Not registered20 (6.1%)0/20 (0%)
      Results availability
       Full text publication of primary results available256 (78.5%)110/256 (43.0%)
       Primary results available as conference abstract and poster8 (2.5%)0/8 (0%)
      Primary results not available62 (19.0%)8/62 (12.9%)
      Abbreviations: RCT, randomized controlled trial.
      a Others: crossover, split body, and unsure.
      For the 326 included RCTs, we identified 118 (36.2%) publicly available protocols. Most were available as peer-reviewed publications (56 of 118; 47.5%) or as a supplementary file with the primary results (48 of 118; 40.7%) (Table 2). When available, 90.9% (100 of 110) of the protocols were referenced by the primary result publication: 55.9% (66 of 118) of available protocols were provided through a link in a clinical trial registry.
      Table 2Forms of protocol availability and whether they were linked in a trial registry
      AvailabilityN (%)
      Total number of protocols available118 (36.2%)
       Protocol as peer-reviewed publication56/118 (47.5%)
       Protocol as a supplementary file with the primary result publication48/118 (40.7%)
       PDF on a trial registry12/118 (10.2%)
       Other type of protocol availability
      2 as PDF on Google scholar or website.
      2/118 (1.7%)
      Protocol linked
      Protocols can be linked to both the registry and the publication of results (n = 50).
       Protocol linked as PDF in a clinical trial registry66/118 (55.9%)
       Protocol linked in result publication
      118 (total number of protocols) −8 (protocols available without publication) = 110 available publications.
      100/110 (90.9%)
       Protocols without publication8/118 (6.8%)
      Abbreviations: PDF, Portable Document Format.
      Supplementary material protocols by journal: The New England Journal of Medicine = 27 of 48 (56.3%), The Lancet Oncology = 6 of 48 (12.5%), Journal of mAmerican Medical Association = 3 of 48 (6.3%), Journal of Clinical Oncology = 3 of 48 (6.3%), and other = 9 of 48 (18.8%).
      a Protocols can be linked to both the registry and the publication of results (n = 50).
      b 2 as PDF on Google scholar or website.
      c 118 (total number of protocols) −8 (protocols available without publication) = 110 available publications.
      Larger sample size (n > 500) and investigator-sponsored trials were associated with increased odds of protocol availability (Table 3). Increased sample size showed evidence for a dose effect, in which each category was associated with an increased proportion of available protocols. The availability of study protocols between drug and nondrug trials was comparable between groups. Among the 118 publicly available protocols, 31 (26.3%) corresponded to trials published in Journal of American Medical Association (JAMA) and New England Journal of Medicine (NEJM) which require protocols to be included with all trial submissions (although 1 trial in JAMA did not provide a protocol). The remaining 88 (74.6%) protocols were associated with trials published in journals that do not require investigators to include a protocol with their trial submission.
      Table 3Trial characteristics associated with protocol availability in logistic regression
      Characteristics
      Reference values: sample size <100, multi-center trials, investigator-initiated trials and drug trials.
      Available (n = 118)Not available (n = 208)UnivariableMultivariable
      OR95% CIP valueOR95% CIP value
      Sample size <10014 (11.9%)59 (28.4%)ReferenceReference
      100–50045 (38.1%)106 (51.0%)1.790.93–3.630.0931.830.91–3.830.09
      >50059 (50%)43 (20.7%)5.782.93–12.02<0.0015.902.75–13.31<0.001
      Multicenter (vs. single center)106 (89.8%)160 (76.9%)2.651.38–5.440.0052.010.92–4.620.087
      Investigator (vs. industry) sponsorship56 (47.5%)91 (43.8%)1.160.74–1.830.5181.991.11–3.590.021
      Drug (vs. non-drug) intervention77 (65.3%)130 (62.5%)1.130.71–1.810.6190.920.51–1.680.788
      Abbreviations: OR, odds ratio; CI, confidence interval.
      a Reference values: sample size <100, multi-center trials, investigator-initiated trials and drug trials.
      Most protocols (101 of 110, 91.8%) were available after the start of the trial (i.e., after the enrollment of the first patient; RTR >0). Only 1 protocol (1 of 110, 0.9%) was made available before enrollment of the first patient (RTR <0), and 2.7% of trial protocols (3 of 110) were made available after publication of the primary results (RTR >1). Protocols were typically made available shortly before publication of the primary results of the RCT (median RTR = 0.90 [IQR = 0.43, 1.00]) but were made available earlier in investigator-sponsored trials (Fig. 1, Table 4). Results from our sensitivity analysis show that industry trials often make their protocols available later as supplementary material (see appendix).
      Figure thumbnail gr1
      Fig. 1Relative time ratio (RTR) of protocol availability by the sponsor. RTR = days from the start of the trial to protocol publication/days from the start of the trial to primary result publication. RTR 0 = start of trial. RTR 1 = publication of trial results. — median RTR.
      Table 4Relative time ratio to protocol availability by categories
      CharacteristicsCount (%) n = 110∗Median RTR [IQR]
      Sample size
       <10011 (10.0)0.55 [0.31, 0.89]
       100–50040 (36.4)0.69 [0.27, 1.00]
       >50059 (53.6)0.96 [0.57, 1.00]
      Number of centers
       Single center9 (8.2)0.41 [0.27, 0.65]
       Multicenter101 (91.8)0.93 [0.46, 1.00]
      Sponsorship
       Investigator51 (46.4)0.60 [0.26, 0.94]
       Industry59 (53.6)1.00 [0.69, 1.00]
      Intervention
       Drug74 (67.2)1.00 [0.66, 1.00]
       Non-drug36 (32.7)0.46 [0.26, 0.91]
      Country of trial approval
       Switzerland50 (45.5)0.93 [0.46, 1.00]
       United Kingdom33 (30.0)0.67 [0.29, 1.00]
       Germany8 (7.3)1.00 [0.64, 1.00]
       Canada19 (17.3)0.94 [0.42, 1.00]
      Abbreviations: RTR, relative time ratio; IQR, interquartile range.
      RTR = days from the start of the trial to protocol publication/days from the start of the trial to primary result publication.
      n = 110∗ = Trials with available protocol and published results.
      RTR <0 = protocol available before the start of the trial.
      RTR >1 = protocol available after primary result publication.
      RTR 0–1 = protocol available during trial.

      4. Discussion

      This meta-research study determined that protocols were only made publicly available for about a third of RCTs in our sample. Larger sample size and investigator sponsorship were associated with increased odds of protocol availability. Moreover, protocols were typically made available shortly before the publication of the primary results, and most industry-initiated trials only made their protocols available at the same time as publication of trial results.
      Previous research corroborates our results of low protocol availability with a similar proportion found in a study of nonpharmacological RCTs (48 of 133, 36.1%) [
      • Sender D.
      • Clark J.
      • Hoffmann T.C.
      Analysis of articles directly related to randomized trials finds poor protocol availability and inconsistent linking of articles.
      ]. In another study examining 261 manuscripts submitted to The Lancet, 250 trials (96%) included a protocol with their submission; however, only 36% made the protocol publicly available (95 of 261, 36%) [
      • Lucey M.
      • Clark J.
      • Glasziou P.
      Public availability of trial protocols.
      ]. Contrary to our findings, Spence et al. determined much higher protocol availability (299 of 364, 82%) in a sample of RCTs published in the top five general medicine journals [
      • Spence O.
      • Hong K.
      • Onwuchekwa Uba R.
      • Doshi P.
      Availability of study protocols for randomized trials published in high-impact medical journals: a cross-sectional analysis.
      ]. The availability of study protocols varied depending on the journal, ranging from 50% (8 of 16) in British Medical Journal (required by the journal to make protocol available since September 2014 []) and more than 95% in NEJM and JAMA [
      • Spence O.
      • Hong K.
      • Onwuchekwa Uba R.
      • Doshi P.
      Availability of study protocols for randomized trials published in high-impact medical journals: a cross-sectional analysis.
      ]. NEJM (since September 2012 [
      • Drazen J.M.
      Believe the data.
      ]) and JAMA (date unclear) require investigators to make their protocols available in order to publish the primary results in the journal. Considering that only one RCT in our sample published in JAMA did not provide a protocol, this may highlight how stricter requirements by journals may prove as a suitable measure to improve protocol availability [
      • Li T.
      • Boutron I.
      • Salman R.A.S.
      • Cobo E.
      • Flemyng E.
      • Grimshaw J.M.
      • et al.
      Review and publication of protocol submissions to trials – what have we learned in 10 years?.
      ,
      • Chan A.W.
      • Hróbjartsson A.
      Promoting public access to clinical trial protocols: challenges and recommendations.
      ]. However, some journals have expressed reluctance to implementing such standards as it may pose an unnecessary burden for investigators conducting smaller trials [
      • Lucey M.
      • Clark J.
      • Glasziou P.
      Public availability of trial protocols.
      ]. Given that it is mandatory to submit a protocol to the ethics committees and other regulatory authorities for approval, this argument does not appear to outweigh the benefits of having protocols available for all conducted RCTs. Apart from initiatives from journals, funding agencies such as the National Institutes of Health (NIH) in the United States are promoting protocol availability through policies requiring investigators to make their protocols available in a trial registry together with the trial results [
      NIH policy on the dissemination of NIH-funded clinical trial information.
      ].
      Corroborating our results, Spence et al. found that protocols were typically made available toward the end of the trial [
      • Spence O.
      • Hong K.
      • Onwuchekwa Uba R.
      • Doshi P.
      Availability of study protocols for randomized trials published in high-impact medical journals: a cross-sectional analysis.
      ]. Spence et al., however, only included protocols published in journals (not considering other formats), thus resulting in overall earlier protocol availability compared to our results. Our study showed earlier accessibility in investigator-sponsored trials and in RCTs with smaller sample sizes, which may be explained by the fact that academic studies benefit more from additional publications generated through protocols [
      • Logisitics B.A.R.
      6 things that differ between industry-initiated and academic clinical trials. Bay area research logistics.
      ]. Although similar proportions of protocols were found as peer-reviewed publications and supplementary materials, the latter were by definition only made available with the primary results, therefore leading to overall later availability. This may also explain why trials with larger sample sizes were made available later, since many large trials are industry sponsored, are published in high-impact journals, and thus have their protocols published as a mandatory supplement in JAMA or NEJM, for instance. Although peer-reviewed feedback for protocols fosters methodological integrity and boosts public awareness and trial trustworthiness [
      • Chan A.W.
      • Hróbjartsson A.
      Promoting public access to clinical trial protocols: challenges and recommendations.
      ], the publication process is associated with lengthy waiting periods, making timely availability difficult. Other platforms such as registries, preprint servers and preregistration platforms, and registered reports may help facilitate earlier availability of trial protocols [
      • Nosek B.A.
      • Ebersole C.R.
      • DeHaven A.C.
      • Mellor D.T.
      The preregistration revolution.
      ,
      • Hardwicke T.E.
      • Ioannidis J.P.A.
      Mapping the universe of registered reports.
      ,
      • Dickersin K.
      Registering clinical trials.
      ,
      • DeVito N.J.
      • Goldacre B.
      Trends and variation in data quality and availability on the European Union Clinical Trials Register: a cross-sectional study.
      ,
      • Bortolini M.A.T.
      Registering a clinical trial.
      ,
      • Klein M.
      • Broadwell P.
      • Farb S.E.
      • Grappone T.
      Comparing published scientific journal articles to their pre-print versions.
      ]. Previous research suggests that trial registries constitute optimal centralized platforms for timely protocol availability [
      • Sender D.
      • Clark J.
      • Hoffmann T.C.
      Analysis of articles directly related to randomized trials finds poor protocol availability and inconsistent linking of articles.
      ,
      • Chan A.W.
      • Hróbjartsson A.
      Promoting public access to clinical trial protocols: challenges and recommendations.
      ,
      • Altman D.G.
      • Furberg C.D.
      • Grimshaw J.M.
      • Rothwell P.M.
      Lead editorial: trials – using the opportunities of electronic publishing to improve the reporting of randomised trials.
      ]. However, only 10% of the included RCTs made their protocol available through a trial registry. Generally, RCTs can take a long time to complete. Making protocols available earlier (i.e., before the investigators analyze their data) and therefore prespecifying how the data will be analyzed may improve the trustworthiness of study results. Furthermore, early sharing of study protocols allows for detailed discussions of methodological procedures, increases transparency, lowers duplication of research, and increases opportunities for collaboration between interested researchers.
      To our knowledge, ours is the first study to quantify protocol availability, as well as timing, in a large and generalizable sample and to explore factors associated with higher availability. Our study has several limitations: first, the sample size was low in some categories (countries, design) which reduce our ability to explore for differences. Second, we included trials approved in 2012 to provide sufficient time for protocols to be made available. As there has been increasing pressure over time to making protocols publicly available [
      • Li T.
      • Boutron I.
      • Salman R.A.S.
      • Cobo E.
      • Flemyng E.
      • Grimshaw J.M.
      • et al.
      Review and publication of protocol submissions to trials – what have we learned in 10 years?.
      ], it is possible that availability has improved. We do plan a study with a sample of RCT protocols approved by ethics committees in 2016 to assess trends over time. Third, we included RCTs from four different high-income countries, and our findings may not be generalizable to trials conducted in low- and middle-income countries. Fourth, the participating ethics committees outside Switzerland constituted a convenience sample, but to the best of our knowledge, these were in no way particular compared to other ethics committees in Canada, Germany, or the United Kingdom.
      In conclusion, only about one-third of RCTs in our cohort made a protocol publicly available, despite consensus in the scientific community that doing so improved transparency, accessibility, and reporting of RCTs. Protocols were typically made available shortly before the publication of the primary results; however, industry-initiated trials were much more likely to publish protocols after data analysis. Larger sample size and investigator sponsorship were associated with increased odds of protocol availability.
      Increased efforts should be made to improve early trial protocol availability, for example, through clinical trial registries or mandatory requirements by journals, funders, ethics committees, or other authorities.

      CRediT authorship contribution statement

      Christof Manuel Schönenberger: Conceptualization, Investigation, Data curation, Writing – original draft. Alexandra Griessbach: Conceptualization, Formal analysis, Investigation, Data curation, Writing – original draft, Visualization. Ala Taji Heravi: Investigation. Dmitry Gryaznov: Investigation. Viktoria L. Gloy: Investigation. Szimonetta Lohner: Investigation. Katharina Klatte: Investigation. Nilabh Ghosh: Investigation. Hopin Lee: Investigation. Anita Mansouri: Investigation. Ioana R. Marian: Investigation. Ramon Saccilotto: Investigation. Edris Nury: Investigation. Jason W. Busse: Investigation. Belinda von Niederhäusern: Investigation. Dominik Mertz: Investigation. Anette Blümle: Investigation. Ayodele Odutayo: Investigation. Sally Hopewell: Investigation. Benjamin Speich: Conceptualization, Methodology, Investigation, Writing – review & editing, Supervision, Project administration. Matthias Briel: Conceptualization, Methodology, Investigation, Writing – review & editing, Supervision, Project administration, Funding acquisition.

      Acknowledgments

      The authors thank all participating research ethics committees from Germany (Freiburg), Switzerland (Basel, Bellinzona, Bern, Geneva, Lausanne, St. Gallen, Frauenfeld, and Zurich), Canada (Hamilton), and the United Kingdom (National Health Service Health Research Authority) for their support and cooperation.

      References

        • Zabor E.C.
        • Kaizer A.M.
        • Hobbs B.P.
        Randomized controlled trials.
        Chest. 2020; 158: S79-S87
        • Chan A.W.
        • Tetzlaff J.M.
        • Gotzsche P.C.
        • Altman D.G.
        • Mann H.
        • Berlin J.A.
        • et al.
        SPIRIT 2013 explanation and elaboration: guidance for protocols of clinical trials.
        BMJ. 2013; 346: e7586
        • Idänpään-heikkilä J.E.
        WHO guidelines for good clinical practice (GCP) for trials on pharmaceutical products: responsibilities of the investigator.
        Ann Med. 1994; 26: 89-94
        • Vijayananthan A.
        • Nawawi O.
        The importance of good clinical practice guidelines and its role in clinical trials.
        Biomed Imaging Interv J. 2008; 4
        • Li T.
        • Boutron I.
        • Salman R.A.S.
        • Cobo E.
        • Flemyng E.
        • Grimshaw J.M.
        • et al.
        Review and publication of protocol submissions to trials – what have we learned in 10 years?.
        Trials. 2017; 18: 34
        • Sender D.
        • Clark J.
        • Hoffmann T.C.
        Analysis of articles directly related to randomized trials finds poor protocol availability and inconsistent linking of articles.
        J Clin Epidemiol. 2020; 124: 69-74
        • Hoffmann T.
        • Glasziou P.
        • Beller E.
        • Goldacre B.
        • Chalmers I.
        Focus on sharing individual patient data distracts from other ways of improving trial transparency.
        BMJ. 2017; 22: j2782
        • Chan A.W.
        • Hróbjartsson A.
        Promoting public access to clinical trial protocols: challenges and recommendations.
        Trials. 2018; 19: 116
        • Altman D.G.
        • Furberg C.D.
        • Grimshaw J.M.
        • Rothwell P.M.
        Lead editorial: trials – using the opportunities of electronic publishing to improve the reporting of randomised trials.
        Trials. 2006; 7: 6
        • Altman D.G.
        • Furberg C.D.
        • Grimshaw J.M.
        • Shanahan D.R.
        Linked publications from a single trial: a thread of evidence.
        Trials. 2014; 15: 369
        • Chan A.W.
        • Hrobjartsson A.
        • Jorgensen K.J.
        • Gotzsche P.C.
        • Altman D.G.
        Discrepancies in sample size calculations and data analyses reported in randomised trials: comparison of publications with protocols.
        BMJ. 2008; 337: a2299
        • Mayo-Wilson E.
        • Li T.
        • Fusco N.
        • Bertizzolo L.
        • Canner J.K.
        • Cowley T.
        • et al.
        Cherry-picking by trialists and meta-analysts can drive conclusions about intervention efficacy.
        J Clin Epidemiol. 2017; 91: 95-110
        • Chan A.W.
        • Pello A.
        • Kitchen J.
        • Axentiev A.
        • Virtanen J.I.
        • Liu A.
        • et al.
        Association of trial registration with reporting of primary outcomes in protocols and publications.
        JAMA. 2017; 318: 1709
        • Mathieu S.
        Comparison of registered and published primary outcomes in randomized controlled trials.
        JAMA. 2009; 302: 977
        • Dwan K.
        • Altman D.G.
        • Clarke M.
        • Gamble C.
        • Higgins J.P.T.
        • Sterne J.A.C.
        • et al.
        Evidence for the selective reporting of analyses and discrepancies in clinical trials: a systematic review of cohort studies of clinical trials.
        PLoS Med. 2014; 11: e1001666
        • Chalmers I.
        • Glasziou P.
        Avoidable waste in the production and reporting of research evidence.
        Lancet. 2009; 374: 86-89
        • Moher D.
        • Liberati A.
        • Tetzlaff J.
        • Altman D.G.
        • The PRISMA Group
        Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement.
        PLoS Med. 2009; 6: e1000097
        • Shamseer L.
        • Moher D.
        • Clarke M.
        • Ghersi D.
        • Liberati A.
        • Petticrew M.
        • et al.
        Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015: elaboration and explanation.
        BMJ. 2015; 349: g7647
        • Nosek B.A.
        • Ebersole C.R.
        • DeHaven A.C.
        • Mellor D.T.
        The preregistration revolution.
        Proc Natl Acad Sci. 2018; 115: 2600-2606
        • Hardwicke T.E.
        • Ioannidis J.P.A.
        Mapping the universe of registered reports.
        Nat Hum Behav. 2018; 2: 793-796
        • Ioannidis J.P.A.
        Why most published research findings are false.
        PLoS Med. 2005; 2: e124
        • Uppstad P.H.
        A rationale for publishing peer-reviewed study protocols in the Nordic Journal of Literacy Research to increase scientific rigour.
        Nordic J Literacy Research. 2020; 6: 1-5
        • Errington T.M.
        • Denis A.
        • Perfito N.
        • Iorns E.
        • Nosek B.A.
        Challenges for assessing replicability in preclinical cancer biology.
        eLife. 2021; 10: e67995
        • Lucey M.
        • Clark J.
        • Glasziou P.
        Public availability of trial protocols.
        Lancet. 2017; 390: e54-e55
        • Spence O.
        • Hong K.
        • Onwuchekwa Uba R.
        • Doshi P.
        Availability of study protocols for randomized trials published in high-impact medical journals: a cross-sectional analysis.
        Clin Trials. 2020; 17: 99-105
        • Gryaznov D.
        • Odutayo A.
        • von Niederhäusern B.
        • Speich B.
        • Kasenda B.
        • Ojeda-Ruiz E.
        • et al.
        Rationale and design of repeated cross-sectional studies to evaluate the reporting quality of trial protocols: the adherence to SPIrit REcommendations (ASPIRE) study and associated projects.
        Trials. 2020; 21: 896
        • Speich B.
        • Gryaznov D.
        • Busse J.W.
        • Gloy V.L.
        • Lohner S.
        • Klatte K.
        • et al.
        Nonregistration, discontinuation, and nonpublication of randomized trials: a repeated metaresearch analysis.
        PLoS Med. 2022; 19: e1003980
      1. Resources for authors the BMJ.
        (Available at)
        • Drazen J.M.
        Believe the data.
        N Engl J Med. 2012; 367: 1152-1153
      2. NIH policy on the dissemination of NIH-funded clinical trial information.
        (Available at)
        • Logisitics B.A.R.
        6 things that differ between industry-initiated and academic clinical trials. Bay area research logistics.
        (Available at)
        • Dickersin K.
        Registering clinical trials.
        JAMA. 2003; 290: 516
        • DeVito N.J.
        • Goldacre B.
        Trends and variation in data quality and availability on the European Union Clinical Trials Register: a cross-sectional study.
        Clin Trials. 2022; 19: 172-183
        • Bortolini M.A.T.
        Registering a clinical trial.
        Int Urogynecol J. 2017; 28: 803-804
        • Klein M.
        • Broadwell P.
        • Farb S.E.
        • Grappone T.
        Comparing published scientific journal articles to their pre-print versions.
        Int J Digit Libr. 2019; 20: 335-350