Original Article| Volume 79, P130-139, November 2016

Randomized trials are frequently fragmented in multiple secondary publications

  • Shanil Ebrahim
    Stanford Prevention Research Center, Department of Medicine, Stanford University, 1265 Welch Road, Stanford, CA 94305, USA

    Meta-Research Innovation Center at Stanford (METRICS), School of Medicine, Stanford University, 1265 Welch Road, Stanford, CA 94305, USA

    Department of Clinical Epidemiology & Biostatistics, McMaster University, 1200 Main Street West, Hamilton, Ontario L8S 4K1, Canada

    Department of Anesthesia, McMaster University, 1200 Main Street West, Hamilton, Ontario L8S 4K1, Canada

    Prevention Lab and Systematic Overviews through advancing Research Technology (SORT), SickKids Research Institute, Hospital for Sick Children, 555 University Avenue, Toronto, Ontario M5G 1X8, Canada
    Search for articles by this author
  • Luis Montoya
    Department of Trauma and Orthopaedics, University Health Network, 399 Bathurst Street, Toronto, Ontario M5T 2S8, Canada
    Search for articles by this author
  • Mostafa Kamal el Din
    Department of Clinical Epidemiology & Biostatistics, McMaster University, 1200 Main Street West, Hamilton, Ontario L8S 4K1, Canada
    Search for articles by this author
  • Zahra N. Sohani
    Department of Clinical Epidemiology & Biostatistics, McMaster University, 1200 Main Street West, Hamilton, Ontario L8S 4K1, Canada

    Population Genomics Program, Faculty of Health Sciences, McMaster University, 1200 Main Street West, Hamilton, Ontario L8S 4K1, Canada

    Faculty of Medicine, 1 King's College Circle, University of Toronto, Toronto, Ontario M5S 1A8, Canada
    Search for articles by this author
  • Arnav Agarwal
    Faculty of Medicine, 1 King's College Circle, University of Toronto, Toronto, Ontario M5S 1A8, Canada
    Search for articles by this author
  • Sheena Bance
    Department of Applied Psychology and Human Development, Ontario Institute for Studies in Education, 252 Bloor Street West, Toronto, Ontario M5S 1V6, Canada
    Search for articles by this author
  • Juliann Saquib
    Department of Family and Community Medicine, Qassim University, Buraydah, Saudi Arabia
    Search for articles by this author
  • Nazmus Saquib
    College of Medicine, Sulaiman AlRajhi Colleges, Al Bukayriyah, Saudi Arabia
    Search for articles by this author
  • John P.A. Ioannidis
    Corresponding author. Tel.: (905) 525-9140; fax: (905) 524-3841.
    Stanford Prevention Research Center, Department of Medicine, Stanford University, 1265 Welch Road, Stanford, CA 94305, USA

    Meta-Research Innovation Center at Stanford (METRICS), School of Medicine, Stanford University, 1265 Welch Road, Stanford, CA 94305, USA

    Department of Health Research and Policy, Stanford University School of Medicine, 150 Governor's Ln, Stanford, CA 94305, USA

    Department of Statistics, Stanford University School of Humanities and Sciences, Building 1, Main Quad, Stanford, CA 94305, USA
    Search for articles by this author



      To assess the frequency and features of secondary publications of randomized controlled trials (RCTs).

      Study Design and Setting

      For 191 RCTs published in high-impact journals in 2009, we searched for secondary publications coauthored by at least one same author of the primary trial publication. We evaluated the probability of having secondary publications, characteristics of the primary trial publication that predict having secondary publications, types of secondary analyses conducted, and statistical significance of those analyses.


      Of 191 primary trials, 88 (46%) had a total of 475 secondary publications by 2/2014. Eight trials had >10 (up to 51) secondary publications each. In multivariable modeling, the risk of having subsequent secondary publications increased 1.32-fold (95% CI 1.05–1.68) per 10-fold increase in sample size, and 1.71-fold (95% CI 1.19–2.45) in the presence of a design article. In a sample of 197 secondary publications examined in depth, 193 tested different hypotheses than the primary publication. Of the 193, 43 tested differences between subgroups, 85 assessed predictive factors associated with an outcome of interest, 118 evaluated different outcomes than the original article, 71 had differences in eligibility criteria, and 21 assessed different durations of follow-up; 176 (91%) presented at least one analysis with statistically significant results.


      Approximately half of randomized trials in high-impact journals have secondary publications published with a few trials followed by numerous secondary publications. Almost all of these publications report some statistically significant results.


      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


      Subscribe to Journal of Clinical Epidemiology
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


        • Chalmers T.C.
        • Smith Jr., H.
        • Blackburn B.
        • Silverman B.
        • Schroeder B.
        • Reitman D.
        • et al.
        A method for assessing the quality of a randomized control trial.
        Control Clin Trials. 1981; 2: 31-49
        • Sanson-Fisher R.W.
        • Bonevski B.
        • Green L.W.
        • D'Este C.
        Limitations of the randomized controlled trial in evaluating population-based health interventions.
        Am J Prev Med. 2007; 33: 155-161
        • Hopewell S.
        • Clarke M.J.
        • Stewart L.
        • Tierney J.
        Time to publication for results of clinical trials.
        (Review) The Cochrane Collaboration, 2008
        • Furberg C.D.
        • Friedman L.M.
        Approaches to data analyses of clinical trials.
        Prog Cardiovasc Dis. 2012; 54: 330-334
      1. NIH Data Sharing Policy and Implementation Guidance, 2013. Available at Accessed January 2015.

        • Doshi P.
        • Goodman S.N.
        • Ioannidis J.P.
        Raw data from clinical trials: within reach?.
        Trends Pharmacol Sci. 2013; 34: 645-647
        • Institute of Medicine of the National Academies
        Sharing clinical trial data: maximizing benefits, minimizing risk.
        Institute of Medicine, Washington DC2015
        • Doshi P.
        • Jefferson T.
        • Del Mar C.
        The imperative to share clinical study reports: recommendations from the Tamiflu experience.
        PLoS Med. 2012; 9: e1001201
        • Friedman L.M.
        • Furberg C.
        • Demets D.L.
        Fundamentals of clinical trials.
        3rd ed. Springer-Verlag, New York1998
        • Ingram J.R.
        • Grindlay D.J.
        • Williams H.C.
        Problems in the reporting of acne clinical trials: a spot check from the 2009 Annual Evidence Update on Acne Vulgaris.
        Trials. 2010; 11: 77
        • Schein M.
        • Paladugu R.
        Redundant surgical publications: tip of the iceberg?.
        Surgery. 2001; 129: 655-661
        • Tramèr M.R.
        • Reynolds D.J.
        • Moore R.A.
        • McQuay H.J.
        Impact of covert duplicate publication on meta-analysis: a case study.
        BMJ. 1997; 315: 635-640
        • von Elm E.
        • Poglia G.
        • Walder B.
        • Tramèr M.R.
        Different patterns of duplicate publication: an analysis of articles used in systematic reviews.
        JAMA. 2004; 291: 974-980
        • Saquib N.
        • Saquib J.
        • Ioannidis J.P.
        Practices and impact of primary outcome adjustment in randomized controlled trials: meta-epidemiologic study.
        BMJ. 2013; 347: f4313
        • Ebrahim S.
        • Sohani Z.N.
        • Montoya L.
        • Agarwal A.
        • Thorlund K.
        • Mills E.J.
        • et al.
        Reanalyses of randomized clinical trial data.
        JAMA. 2014; 312: 1024-1032
        • Twisk J.W.
        • Smidt N.
        • de Vente W.
        Applied analysis of recurrent events: a practical overview.
        J Epidemiol Community Health. 2005; 59: 706-710
        • Andriole G.L.
        • Crawford E.D.
        • Grubb 3rd, R.L.
        • Buys S.S.
        • Chia D.
        • Church T.R.
        • et al.
        Mortality results from a randomized prostate-cancer screening trial.
        N Engl J Med. 2009; 360: 1310-1319
        • Cummings S.R.
        • San Martin J.
        • McClung M.R.
        • Siris E.S.
        • Eastell R.
        • Reid I.R.
        • et al.
        Denosumab for prevention of fractures in postmenopausal women with osteoporosis.
        N Engl J Med. 2009; 361: 756-765
        • Foster G.D.
        • Borradaile K.E.
        • Sanders M.H.
        • Millman R.
        • Zammit G.
        • Newman A.B.
        • et al.
        A randomized study on the effect of weight loss on obstructive sleep apnea among obese patients with type 2 diabetes: the Sleep AHEAD study.
        Arch Intern Med. 2009; 169: 1619-1626
        • Mehran R.
        • Lansky A.J.
        • Witzenbichler B.
        • Guagliumi G.
        • Peruga J.Z.
        • Brodie B.R.
        • et al.
        Bivalirudin in patients undergoing primary angioplasty for acute myocardial infarction (HORIZONS-AMI): 1-year results of a randomised controlled trial.
        Lancet. 2009; 374: 1149-1159
        • Moss A.J.
        • Hall W.J.
        • Cannom D.S.
        • Klein H.
        • Brown M.W.
        • Daubert J.P.
        • et al.
        Cardiac-resynchronization therapy for the prevention of heart-failure events.
        N Engl J Med. 2009; 361: 1329-1338
        • O'Connor C.M.
        • Whellan D.J.
        • Lee K.L.
        • Keteyian S.J.
        • Cooper L.S.
        • Ellis S.J.
        • et al.
        Efficacy and safety of exercise training in patients with chronic heart failure: HF-ACTION randomized controlled trial.
        JAMA. 2009; 301: 1439-1450
        • Piper M.E.
        • Smith S.S.
        • Schlam T.R.
        • Fiore M.C.
        • Jorenby D.E.
        • Fraser D.
        • et al.
        A randomized placebo-controlled clinical trial of 5 smoking cessation pharmacotherapies.
        Arch Gen Psychiatry. 2009; 66 (Epub 2009 Nov 04): 1253-1262
        • Wallentin L.
        • Becker R.C.
        • Budaj A.
        • Cannon C.P.
        • Emanuelsson H.
        • Held C.
        • et al.
        Ticagrelor versus clopidogrel in patients with acute coronary syndromes.
        N Engl J Med. 2009; 361: 1045-1057
        • Lopes R.D.
        • Dickerson S.
        • Hafley G.
        • Burns S.
        • Tourt-Uhlig S.
        • White J.
        • et al.
        Methodology of a reevaluation of cardiovascular outcomes in the RECORD trial: study design and conduct.
        Am Heart J. 2013; 166: 208-216
        • Bray G.
        • Gregg E.
        • Haffner S.
        • Pi-Sunyer X.F.
        • WagenKnecht L.E.
        • Walkup M.
        • et al.
        Baseline characteristics of the randomised cohort from the Look AHEAD (Action for Health in Diabetes) study.
        Diab Vasc Dis Res. 2006; 3: 202-215
        • Kallmes D.F.
        • Comstock B.A.
        • Gray L.A.
        • Heagerty P.J.
        • Hollingworth W.
        • Turner J.A.
        • et al.
        Baseline pain and disability in the investigational vertebroplasty efficacy and safety trial.
        AJNR Am J Neuroradiol. 2009; 30: 1203-1205
        • Leemans C.R.
        • Tijink B.M.
        • Langendijk J.A.
        • Andry G.
        • Hamoir M.
        • Lefebvre J.L.
        Quality assurance in head and neck surgical oncology: EORTC 24954 trial on larynx preservation.
        Eur J Surg Oncol. 2013; 39: 1013-1018
        • Home P.D.
        • Pocock S.J.
        • Beck-Nielsen H.
        • Curtis P.S.
        • Gomis R.
        • Hanefeld M.
        • et al.
        • RECORD Study Team
        Rosiglitazone evaluated for cardiovascular outcomes in oral agent combination therapy for type 2 diabetes (RECORD): a multicentre, randomised, open-label trial.
        Lancet. 2009; 373: 2125-2135
        • Milgrom H.
        • Fowler-Taylor A.
        • Vidaurre C.F.
        • Jayawardene S.
        Safety and tolerability of omalizumab in children with allergic (IgE-mediated) asthma.
        Curr Med Res Opin. 2011; 27: 163-169
        • Regueiro M.
        • El-Hachem S.
        • Kip K.E.
        • Schraut W.
        • Baidoo L.
        • Watson A.
        • et al.
        Postoperative infliximab is not associated with an increase in adverse events in Crohn's disease.
        Dig Dis Sci. 2011; 56: 3610-3615
        • Freedman R.A.
        • Pitcher B.
        • Keating N.L.
        • Ballman K.V.
        • Mandelblatt J.
        • Kornblith A.B.
        • et al.
        Cognitive function in older women with breast cancer treated with standard chemotherapy and capecitabine on Cancer and Leukemia Group B 49907.
        Breast Cancer Res Treat. 2013; 139: 607-616
        • Sugiura R.
        • Ogawa H.
        • Oka T.
        • Koyanagi R.
        • Hagiwara N.
        Candesartan-Based therapy and risk of cancer in patients with systemic hypertension (Heart Institute of Japan Candesartan Randomized Trial for Evaluation in Coronary Artery Disease [HIJ-CREATE] substudy).
        Am J Cardiol. 2012; 109: 576-580
        • Home P.D.
        • Pocock S.J.
        • Beck-Nielsen H.
        • Gomis R.
        • Hanefeld M.
        • Jones N.P.
        • et al.
        Rosiglitazone evaluated for cardiovascular outcomes–an interim analysis.
        N Engl J Med. 2007; 357: 28-38
        • Arts M.
        • Brand R.
        • van der Kallen B.
        • Lycklama à Nijeholt G.
        • Peul W.
        Does minimally invasive lumbar disc surgery result in less muscle injury than conventional surgery? A randomized controlled trial.
        Eur Spine J. 2011; 20: 51-57
        • Ruwald A.C.
        • Sood N.
        • Ruwald M.H.
        • Jons C.
        • Clyne C.A.
        • McNitt S.
        • et al.
        Frequency of inappropriate therapy in patients implanted with dual- versus single-chamber ICD devices in the ICD arm of MADIT-CRT.
        J Cardiovasc Electrophysiol. 2013; 24: 672-679
        • van den Akker M.E.
        • Arts M.P.
        • van den Hout W.B.
        • Brand R.
        • Koes B.W.
        • Peul W.C.
        Tubular diskectomy vs conventional microdiskectomy for the treatment of lumbar disk-related sciatica: cost utility analysis alongside a double-blind randomized controlled trial.
        Neurosurgery. 2011; 69: 829-836
        • Soares M.O.
        • Iglesias C.P.
        • Bland J.M.
        • Cullum N.
        • Dumville J.C.
        • Nelson E.A.
        • et al.
        Cost effectiveness analysis of larval therapy for leg ulcers.
        BMJ. 2009; 338: b825
        • Barlinn K.
        • Tsivgoulis G.
        • Molina C.A.
        • Alexandrov D.A.
        • Schafer M.E.
        • Alleman J.
        • et al.
        Exploratory analysis of estimated acoustic peak rarefaction pressure, recanalization, and outcome in the transcranial ultrasound in clinical sonothrombolysis trial.
        J Clin Ultrasound. 2013; 41: 354-360
        • Fiuzat M.
        • Schulte P.J.
        • Felker M.
        • Ahmad T.
        • Neely M.
        • Adams K.F.
        • et al.
        Relationship between galectin-3 levels and mineralocorticoid receptor antagonist use in heart failure: analysis from HF-ACTION.
        J Card Fail. 2014; 20: 38-44
        • Hopewell S.
        Reporting characteristics of non-primary publications of results of randomized trials: a cross-sectional review.
        Trials. 2013; 14: 240
        • Fanelli D.
        Do pressures to publish increase scientists' bias? An empirical support from US States Data.
        PLoS One. 2010; 5: e10271
        • Fanelli D.
        “Positive” results increase down the Hierarchy of the Sciences.
        PLoS One. 2010; 5: e10068
        • Thompson S.G.
        • Higgins J.P.
        Treating individuals 4: can meta-analysis help target interventions at individuals most likely to benefit?.
        Lancet. 2005; 365: 341-346
        • Rothwell P.M.
        Treating individuals 2. Subgroup analysis in randomised controlled trials: importance, indications, and interpretation.
        Lancet. 2005; 365: 176-186
        • Wang R.
        • Lagakos S.W.
        • Ware J.H.
        • Hunter D.J.
        • Drazen J.M.
        Statistics in medicine–reporting of subgroup analyses in clinical trials.
        N Engl J Med. 2007; 357: 2189-2194
        • Lagakos S.W.
        The challenge of subgroup analyses–reporting without distorting.
        N Engl J Med. 2006; 354: 1667-1669
        • Tzoulaki I.
        • Siontis K.C.
        • Evangelou E.
        • Ioannidis J.P.
        Bias in associations of emerging biomarkers with cardiovascular disease.
        JAMA Intern Med. 2013; 173: 664-671
        • Kyzas P.A.
        • Denaxa-Kyza D.
        • Ioannidis J.P.
        Almost all articles on cancer prognostic markers report statistically significant results.
        Eur J Cancer. 2007; 43: 2559-2579
        • Chan A.W.
        • Altman D.G.
        Identifying outcome reporting bias in randomised trials on PubMed: review of publications and survey of authors.
        BMJ. 2005; 330: 753
        • Chan A.W.
        • Krleza-Jerić K.
        • Schmid I.
        • Altman D.G.
        Outcome reporting bias in randomized trials funded by the Canadian Institutes of Health Research.
        CMAJ. 2004; 171: 735-740
        • Chan A.W.
        • Hróbjartsson A.
        • Haahr M.T.
        • Gøtzsche P.C.
        • Altman D.G.
        Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles.
        JAMA. 2004; 291: 2457-2465
      2. Women's Health Initiative, 2015. Available at https:// Accessed January 2015.

        • Altman D.G.
        • Furberg C.D.
        • Grimshaw J.M.
        • Shanahan D.R.
        Linked publications from a single trial: a thread of evidence.
        Trials. 2014; 15: 369