Head-to-head randomized trials are mostly industry sponsored and almost always favor the industry sponsor

Open AccessPublished:February 07, 2015DOI:https://doi.org/10.1016/j.jclinepi.2014.12.016

      Abstract

      Objectives

      To map the current status of head-to-head comparative randomized evidence and to assess whether funding may impact on trial design and results.

      Study Design and Setting

      From a 50% random sample of the randomized controlled trials (RCTs) published in journals indexed in PubMed during 2011, we selected the trials with ≥100 participants, evaluating the efficacy and safety of drugs, biologics, and medical devices through a head-to-head comparison.

      Results

      We analyzed 319 trials. Overall, 238,386 of the 289,718 randomized subjects (82.3%) were included in the 182 trials funded by companies. Of the 182 industry-sponsored trials, only 23 had two industry sponsors and only three involved truly antagonistic comparisons. Industry-sponsored trials were larger, more commonly registered, used more frequently noninferiority/equivalence designs, had higher citation impact, and were more likely to have “favorable” results (superiority or noninferiority/equivalence for the experimental treatment) than nonindustry-sponsored trials. Industry funding [odds ratio (OR) 2.8; 95% confidence interval (CI): 1.6, 4.7] and noninferiority/equivalence designs (OR 3.2; 95% CI: 1.5, 6.6), but not sample size, were strongly associated with “favorable” findings. Fifty-five of the 57 (96.5%) industry-funded noninferiority/equivalence trials got desirable “favorable” results.

      Conclusion

      The literature of head-to-head RCTs is dominated by the industry. Industry-sponsored comparative assessments systematically yield favorable results for the sponsors, even more so when noninferiority designs are involved.

      Keywords

      1. Introduction

      What is new?

         Key findings

      • Most head-to-head comparative evidence is procured by industry-sponsored trials. Head-to-head industry trials tend to be larger, more frequently registered and to yield “favorable” results for the experimental treatment. They also use noninferiority/equivalence designs more frequently than nonindustry-sponsored trials. Industry funding and noninferiority/equivalence designs are strong predictors of “favorable” findings, and when they coexist, almost all trials get desirable “favorable” results.

         What this study adds to what was known?

      • It has been speculated whether companies tend to avoid head-to-head comparisons and to avoid jeopardizing their market share by unfavorable results. However, we found a strong dominance of the industry in the influential agenda of head-to-head comparisons and a high prevalence of results that are favorable for the sponsoring companies in this literature.

         What is the implication and what should change now?

      • Given the pivotal importance of head-to-head trials in generating influential comparative evidence, consideration should be given to allowing the conduct of more large trials of comparative effectiveness and safety under the control of nonprofit entities.
      It is becoming increasingly common to have multiple treatment options for managing many medical conditions. Most randomized trials to date have evaluated the effectiveness and safety of active interventions against inactive controls (placebo, no treatment, or standard of care) [
      • Ioannidis J.P.
      Indirect comparisons: the mesh and mess of clinical trials.
      ]. However, comparative evidence from head-to-head randomized comparisons may be indispensable to capture the relative benefits and harms of alternative interventions [
      • Owens D.K.
      • Lohr K.N.
      • Atkins D.
      • Treadwell J.R.
      • Reston J.T.
      • Bass E.B.
      • et al.
      AHRQ series paper 5: grading the strength of a body of evidence when comparing medical interventions—agency for healthcare research and quality and the effective health-care program.
      ,
      • Zentner A.
      • Velasco-Garrido M.
      • Busse R.
      Methods for the comparative evaluation of pharmaceuticals.
      ,
      • Estellat C.
      • Ravaud P.
      Lack of head-to-head trials and fair control arms: randomized controlled trials of biologic treatment for rheumatoid arthritis.
      ].
      Previous empirical assessments of trials on head-to-head comparisons have focused mostly on single domains or specialties, such as cardiovascular medicine and psychiatry [
      • Ridker P.M.
      • Torres J.
      Reported outcomes in major cardiovascular clinical trials funded by for-profit and not-for-profit organizations: 2000-2005.
      ,
      • Bero L.
      • Oostvogel F.
      • Bacchetti P.
      • Lee K.
      Factors associated with findings of published trials of drug-drug comparisons: why some statins appear more efficacious than others.
      ,
      • Heres S.
      • Davis J.
      • Maino K.
      • Jetzinger E.
      • Kissling W.
      • Leucht S.
      Why olanzapine beats risperidone, risperidone beats quetiapine, and quetiapine beats olanzapine: an exploratory analysis of head-to-head comparison studies of second-generation antipsychotics.
      ,
      • Gartlehner G.
      • Morgan L.
      • Thieda P.
      • Fleg A.
      The effect of study sponsorship on a systematically evaluated body of evidence of head-to-head trials was modest: secondary analysis of a systematic review.
      ]. These assessments have suggested that most of these trials are sponsored by the industry. Head-to-head comparisons comprise a very small proportion of the industry-sponsored clinical trials agenda, which is vastly dominated by trials that involve testing only a single product by a single manufacturer sponsor [
      • Lathyris D.N.
      • Patsopoulos N.A.
      • Salanti G.
      • Ioannidis J.P.
      Industry sponsorship and selection of comparators in randomized clinical trials.
      ]. However, given the pivotal importance of head-to-head trials in generating influential comparative evidence [
      • Owens D.K.
      • Lohr K.N.
      • Atkins D.
      • Treadwell J.R.
      • Reston J.T.
      • Bass E.B.
      • et al.
      AHRQ series paper 5: grading the strength of a body of evidence when comparing medical interventions—agency for healthcare research and quality and the effective health-care program.
      ,
      • Zentner A.
      • Velasco-Garrido M.
      • Busse R.
      Methods for the comparative evaluation of pharmaceuticals.
      ], it is important to understand the profile of currently performed head-to-head randomized comparisons and whether their sponsorship has any impact on their design and their results. For randomized trials in general, there are suggestions that industry funding may be associated with more favorable results and conclusions [
      • Sackett D.L.
      • Rosenberg W.M.
      • Gray J.A.
      • Haynes R.B.
      • Richardson W.S.
      Evidence based medicine: what it is and what it isn't.
      ,
      • Stamatakis E.
      • Weiler R.
      • Ioannidis J.P.
      Undue industry influences that distort healthcare research, strategy, expenditure and practice: a review.
      ,
      • DeAngelis C.D.
      The influence of money on medical science.
      ,
      • Patsopoulos N.A.
      • Ioannidis J.P.
      • Analatos A.A.
      Origin and funding of the most frequently cited papers in medicine: database analysis.
      ,
      • Melander H.
      • Ahlqvist-Rastad J.
      • Meijer G.
      • Beermann B.
      Evidence b(i)ased medicine—selective reporting from studies sponsored by pharmaceutical industry: review of studies in new drug applications.
      ,
      • Yank V.
      • Rennie D.
      • Bero L.A.
      Financial ties and concordance between results and conclusions in meta-analyses: retrospective cohort study.
      ,
      • Ioannidis J.P.
      • Greenland S.
      • Hlatky M.A.
      • Khoury M.J.
      • Macleod M.R.
      • Moher D.
      • et al.
      Increasing value and reducing waste in research design, conduct, and analysis.
      ,
      • Ross J.S.
      • Hill K.P.
      • Egilman D.S.
      • Krumholz H.M.
      Guest authorship and ghostwriting in publications related to rofecoxib: a case study of industry documents from rofecoxib litigation.
      ,
      • Hill K.P.
      • Ross J.S.
      • Egilman D.S.
      • Krumholz H.M.
      The ADVANTAGE seeding trial: a review of internal documents.
      ], but also better methodological quality compared with nonindustry-funded trials [
      • Lexchin J.
      • Bero L.A.
      • Djulbegovic B.
      • Clark O.
      Pharmaceutical industry sponsorship and research outcome and quality: systematic review.
      ]. It would be interesting to assess whether the profile and characteristics, types of designs used, and results of head-to-head comparisons in randomized trials are also influenced by their sponsoring. Head-to-head comparisons in particular pose some additional design challenges, such as the choice of superiority or noninferiority/equivalence designs [
      • Garattini S.
      • Bertele V.
      Non-inferiority trials are unethical because they disregard patients' interests.
      ,
      • Powers J.H.
      • Cooper C.K.
      • Lin D.
      • Ross D.B.
      Sample size and the ethics of non-inferiority trials.
      ], which may also affect the final inferences drawn. Moreover, given that two or more active treatments are involved in the comparison, in theory more than one sponsor may be involved and antagonistic situations may ensue if the sponsors are manufacturers of the compared interventions. Preliminary evidence suggests that companies avoid to cosponsor the same trial with such antagonistic comparisons [
      • Lathyris D.N.
      • Patsopoulos N.A.
      • Salanti G.
      • Ioannidis J.P.
      Industry sponsorship and selection of comparators in randomized clinical trials.
      ]. This avoidance may generate fragmented evidence, where little evidence is available on interventions that have been manufactured by different companies. It would be interesting to examine whether this is a common pattern across a large number of head-to-head comparisons.
      To map the current status of head-to-head comparative evidence, here we analyzed a large sample of recently published head-to-head randomized clinical trials covering a wide range of clinical conditions. We evaluated trials with at least 100 participants because these larger trials are the ones that are most likely to have a greater impact on our perceptions of the accumulated evidence and therefore also on medical practice. We specifically focused on the sponsoring of these trials and whether sponsoring by the industry affected the characteristics, design, and findings/conclusions from these studies.

      2. Methods

      2.1 Bibliographic search and inclusion study criteria

      On March 15, 2013, we performed a search in PubMed using the following terms: [Randomized controlled trial OR randomised controlled trial OR randomized trial OR randomised trial] and restricted the results to trials published in any language in 2011. After the methodology by Resnik et al. [
      • Resnik D.B.
      • Peddada S.
      • Brunson Jr., W.
      Research misconduct policies of scientific journals.
      ,
      • Resnik D.B.
      • Patrone D.
      • Peddada S.
      Research misconduct policies of social science journals and impact factor.
      ], we randomly selected 50% of the 20,088 items published during 2011. Each item was assigned a number according to the order of appearance in PubMed. A computer-generated random sequence was then created (from 1 to 20,088 without replacement), and the first 50% items were selected for review. We then screened all trials for inclusion based on the abstract or, if needed, the full text.
      We included only randomized controlled trials (RCTs) evaluating the efficacy and safety of drugs, biologics, or medical devices, in which two or more interventions were directly compared, regardless of whether there were also common backbone treatments (common treatments across study arms). Trials comparing an intervention against placebo as well as comparing different doses or ways of administration of the same intervention were excluded, regardless of whether additional backbone treatments were given to all study arms. Also, we limited our search to trials with sample size ≥100 to avoid the inclusion of smaller studies that may be less influential on evidence-based inferences and their application to medical practice. The threshold of n = 100 is arbitrary and was set a priori.

      2.1.1 Data extraction

      Eligible studies were scrutinized to extract the following data: sample size; study location(s); disease or condition(s); compared interventions; backbone interventions; registration in at least one trial registry among ClinicalTrials.gov, WHO ICTRP, ISRCTN, Australian/New Zealand, and Indian Clinical Trials Registries (if no trial registration number was provided in the article, we manually searched the previously mentioned databases to identify the corresponding trial using the following information: trial sample, interventions, primary and secondary outcomes, and funding sources); design (superiority, noninferiority, equivalence); “favorable” or “unfavorable” trial results for the experimental therapy: a trial was classified as “favorable” if, for at least one main outcome among those defined in the protocol, the experimental therapy was significantly (P < 0.05) better than the standard therapy (in superiority trials), the experimental therapy was not substantially worse than the standard therapy (in noninferiority trials), or the effects of the treatments differed by no more than the equivalence margin (in equivalence trials) [
      • Hopewell S.
      • Loudon K.
      • Clarke M.J.
      • Oxman A.D.
      • Dickersin K.
      Publication bias in clinical trials due to statistical significance or direction of trial results.
      ,
      • Ioannidis J.P.
      Effect of the statistical significance of results on the time to completion and publication of randomized efficacy trials.
      ]; name of the journal publishing the trial, its impact factor value according to the JCR 2012, and number of citations received by each trial in Scopus until March 15, 2013; type of funding source (not reported, governmental and/or other nonprofit organizations, industry). A trial was classified as industry funded if: (1) an explicit acknowledgment of support from private industry was provided in the article, or stated in the acknowledgment section, or (if the trial was registered) declared in the corresponding trial registration record; or (2) at least one author was a company employee, or received grants from the company, or held stocks or options; or (3) the statistical analysis was performed by the company [
      • Lexchin J.
      • Bero L.A.
      • Djulbegovic B.
      • Clark O.
      Pharmaceutical industry sponsorship and research outcome and quality: systematic review.
      ,
      • Lundh A.
      • Sismondo S.
      • Lexchin J.
      • Busuioc O.A.
      • Bero L.
      Industry sponsorship and research outcome.
      ]. In case of multiple sources of funding, a trial was considered industry funded if the points (1) or (2) were met, regardless of the presence of other, nonprofit sources of funding, or the affiliations of the co-authors. We also noted the name(s) of the funding source(s) (if no funding source was listed in the article, we checked the corresponding trial registration record); owner(s) of the products under evaluation (the ones compared and any backbone intervention); authors' affiliation (industry, nonprofit, academic) and name of the affiliation company; and presence (yes/no) of a conflict of interest with the sponsors or owners, as declared in the article. When not explicitly stated in the article, the presence of a conflict of interest was deduced if at least one of the authors was an employee of the same company sponsoring the trial or commercializing one of the products compared or used as backbone intervention.
      Because financial agreements among companies are common, we tried to identify and merge with the mother company any affiliates, subsidiaries, or branches (even with different names), which were seemingly owning or sponsoring different interventions. For each drug, we extracted the ownership at the time of trial publication (2011). When a drug was manufactured by more than one company, we further tried to identify the owners of compared interventions by checking article and/or trial registry entries because most of them reported the trade names (and the manufacturers) of compared interventions. If these data were not available, we retrieved useful information on drugs manufacturer and sponsors by perusing the 2011 Edition of the Physician's Desk Reference (65th Edition, 2011) and/or the British National Formulary (version 62, September 2011). When the owner of an intervention could not be found, we also searched Wikipedia, company Web sites, and drug-related Web sites [
      • Alsheikh-Ali A.A.
      • Qureshi W.
      • Al-Mallah M.H.
      • Ioannidis J.P.
      Public availability of published research data in high-impact journals.
      ]. Four independent researchers extracted the data and discrepancies were discussed to reach consensus.

      2.2 Statistical analysis

      We present descriptive statistics (medians and interquartile range for continuous variables, proportions for discrete variables) on the characteristics of head-to-head comparison trials. We further evaluated all these characteristics in relationship to the type of funding to assess whether industry-sponsored trials were different than nonprofit funded and those where no funding was listed. Descriptive data on trial characteristics are also presented separately for the 12 companies sponsoring the largest number of eligible studies.
      We also estimated the number of RCTs with more than one sponsor and examined whether any trials were funded by companies which were not the owners of the products under evaluation. After the methodology by Lathyris et al. [
      • Lathyris D.N.
      • Patsopoulos N.A.
      • Salanti G.
      • Ioannidis J.P.
      Industry sponsorship and selection of comparators in randomized clinical trials.
      ], we described the network and degree of cosponsorship by companies through a matrix that included the 12 companies with the most prolific trial agenda. Each company was tabulated against each of the others to visualize the number of the RCTs that have been cosponsored. In the network, each company is shown by a node whose diameter is proportional to the number of sponsored trials. Lines connecting two nodes represent cosponsorship between industries; the thickness of each link is proportional to the number of cosponsored trials. Around each node, there are loops whose thickness is proportional to the number of trials that were funded only by the company represented by the respective node.
      Finally, we evaluated in exploratory analyses whether industry funding, larger sample size, and design (noninferiority/equivalence rather than superiority) were associated with “favorable” results in univariate and multivariate logistic regressions. The variables mentioned previously were forced to entry. We also tested whether any of the other recorded variables (affiliation, registration, impact factor, country, intervention type, conflict of interest) was significantly associated with favorable results in the final model, but none remained statistically significant when funding source and study design were adjusted for in the model.
      Statistical significance was defined as a two-sided P-value <0.05, and all analyses were carried out using Stata 11.1 (Stata Corp., College Station, TX, USA, 2011).

      3. Results

      3.1 Characteristics of included trials

      Fig. 1 shows the flow diagram of trial selection process. Among 6,526 potentially eligible reports of RCTs, 498 entailed potentially eligible head-to-head comparisons (7.6%). After the exclusion of the RCTs with <100 participants, 319 head-to-head RCTs were included in the analysis.
      Figure thumbnail gr1
      Fig. 1Flow diagram of the search. Study selection was made applying the criteria below reported in the same order in which they are shown in the Figure. RCT, randomized controlled trial.
      Table 1 lists the main characteristics of the trials (references appear in the Appendix). The median sample size was 300, and only 46 trials exceeded 1,000 participants (6 trials exceeded 10,000 participants). Almost 30% of the 319 trials enrolled participants from multiple countries; 22.9% of the trials were carried out in Western Europe only, 21.3% in Asia, and 19.4% in the United States. Most trials (90.3%) tested the efficacy and/or safety of two or more drugs; biologics were compared in 9.8% of the sample, and no trial compared medical devices. A wide variety of diseases and types of interventions were evaluated. In 85 trials (26.6%), a common backbone treatment was administered to all patients enrolled. Most trials (69.6%) posted the protocol into a registry (ClinicalTrials.gov in 95.4% of the registered studies). Excluding backbone treatments, 249 trials (78.1%) entailed an antagonistic comparison of active interventions owned by different companies. Almost three-quarters (73%) had a superiority design, and a similar proportion (73.7%) reported “favorable” findings. The proportion of “favorable” findings was 68.2% (159 of 233) for superiority designs and 88.2% (75 of 85) for noninferiority/equivalence designs.
      Table 1Characteristics of included trials
      VariablesOverall sample (n = 319)
      Trial sample, median (IQR)300 (446)
      Location of the RCTs, n (%)
       United States62 (19.4)
       Western Europe73 (22.9)
       Asia68 (21.3)
       International
      Trials conducted in countries from different continents.
      93 (29.2)
       Others23 (7.2)
      Context under evaluation, n (%)
       Cardiovascular disorders57 (17.9)
       Cancer44 (13.8)
       Infectious diseases40 (12.5)
       Anesthesiology20 (6.3)
       Diabetes mellitus18 (5.6)
       Others140 (43.9)
      Type of intervention, n (%)
       Drugs288 (90.3)
       Biologics31 (9.7)
      Compared interventions (ATC class), n (%)
       Anti-infectives for systemic use35 (11.0)
       Antineoplastic and immunomodulating agents61 (19.1)
       Nervous system52 (16.3)
       Alimentary tract and metabolism47 (14.7)
       Cardiovascular system38 (11.9)
       Blood and blood forming organs27 (8.5)
       Genitourinary system and sex hormones16 (5.0)
       Antiparasitic products11 (3.5)
       Dermatologicals10 (3.1)
       Musculoskeletal system10 (3.1)
       Respiratory system7 (2.2)
       Sensory organs2 (0.6)
       Systemic hormonal preparations (excluding sex hormones)1 (0.3)
       Various2 (0.6)
      Trials with common backbone treatment, n (%)85 (26.6)
      Registered trials, n (%)222 (69.6)
      Trial design, n (%)
      N = 318 (one trial with unspecified design was excluded from the comparison).
       Superiority233 (73.0)
       Noninferiority73 (22.9)
       Equivalence12 (3.8)
      Trials with favorable results, n (%)235 (73.7)
      Journal impact factor, median (IQR)3.4 (4.1)
      Number of citations, median (IQR)6 (12)
      Type of funding source, n (%)
       Pharmaceutical companies182 (57.0)
       Nonprofit institutions87 (27.3)
       Not reported50 (15.7)
      Trials sponsored by >1 company, n (%)23 (7.2)
      Authors' affiliation, n (%)
       Academic only117 (36.7)
       Both academic and nonprofit64 (20.0)
       Nonprofit only14 (4.4)
       Industry (at least one author)124 (38.9)
      Authors declaring a conflict of interest, n (%)
      Presence of a potential conflict of interest with the sponsors or owners, as declared in the article (see text for details).
      185 (58.0)
      Abbreviations: IQR, interquartile range; RCT, randomized controlled trial; ATC, Anatomical Therapeutic Chemical classification system.
      a Trials conducted in countries from different continents.
      b N = 318 (one trial with unspecified design was excluded from the comparison).
      c Presence of a potential conflict of interest with the sponsors or owners, as declared in the article (see text for details).

      3.2 Funding source

      As reported in Table 1, more than half of the trials had industry sponsoring (57%) and these trials accounted for 238,386 of the 289,718 randomized participants (82.3%). Trials supported by not-for-profit institutions were 27.3% (14.6% of the randomized participants); 15.7% RCTs (3.1% of the randomized participants) did not report a source of funding.
      Only 23 of 182 industry trials (12.6%) were funded by more than one company (Table S1/Appendix at www.jclinepi.com), and only nine (2.8%) were also supported by nonprofit institutions. The great imbalance between cosponsored trials and trials supported by a single company is visualized in Fig. 2, which displays the network of cosponsorship for the 12 most prolific companies.
      Figure thumbnail gr2
      Fig. 2Network of cosponsorship for the 12 most prolific companies. Each company is shown by a node whose diameter is proportional to the number of trials sponsored. Lines represent cosponsorship between companies, with thickness proportional to the number of trials cosponsored. Dashed lines refer to comparison with only one cosponsored trial. The thickness of the autoloops is proportional to the number of trials where the respective company is the unique sponsor. Red circles into each node represent the proportion of trials sponsored (or cosponsored) by each company that showed favorable results to the specific company (corresponding percentages are listed next to each node). BMS, Bristol-Myers Squibb.
      As shown, the thickness of the autoloops (which represent trials sponsored by one company) is very large in proportion to the sparse links between pairs of different companies. Six of the 12 companies had not cosponsored any trial with any of the other major companies. For descriptive purposes, we have reported the main characteristics of the 101 trials sponsored by these 12 companies (Table S2/Appendix at www.jclinepi.com).
      Most of the 23 cosponsored trials (n = 18) were funded by companies, which were also the owners of the products under antagonistic evaluation (Tables S1 and S3/Appendix at www.jclinepi.com): in 11 of the 18, a product was co-owned by the two sponsors; in 8 of the 18, one of the sponsors owned both interventions (the standard of care and the new drug); and in 3 of the 18, one of the sponsors, in addition to the ownership of an active comparator, commercialized a common backbone treatment given to all arms. Only in 3 of 18 trials [PubMed identification codes (PMID)—20890207, 21397567, 21729834], two different companies owned two different active treatments and supported the comparison between them. In the remaining five trials (PMID 21149659, 21388938, 21434995, 21652683, and 21775930), one of the sponsors did not own any of the compared drugs, but it commercialized the common backbone treatment given to all arms. Of the 23 cosponsored trial reports, three were secondary analyses of trials whose primary results had been published previously, and in none of the three, the primary results publication had been sponsored only by a single company.
      Based on authors' affiliation, 38.9% of the trials had at least one author affiliated to industry; 56.7% had been authored by researchers with an academic/nonprofit affiliation (Table 1). Overall, there was some conflict of interest between the authors and the industry in 58.0% of the trials.

      3.3 Trial characteristics by funding source

      As shown in Table 2, industry vs. other trials were significantly more likely to involve international research groups, enrolled a higher number of participants, were more likely to be registered, use noninferiority designs, have “favorable” results, be published on high-impact scientific journals, and were also more cited. The same pattern was seen for industry-sponsored trials vs. the group of nonprofit trials [although the difference was not nominally significant for journal impact factor (P = 0.2), number of citations (P = 0.7), presence of an antagonistic comparison (P = 0.13)] and vs. trials without listed funding. Of the 50 trials without listed funders, only one was done in multiple countries, only one was registered, and the sample size tended to be small, while their impact was generally very limited. Seven of the nine trials cosponsored by both profit and nonprofit institutions showed positive findings (77.8%), in line with the rest of industry trials (P = 0.7).
      Table 2Characteristics of the included trials according to the funding source (n = 319)


      Variables
      For-profit companies (n = 182; 57.0%)Nonprofit institutions only (n = 87; 27.3%)Funding source not reported (n = 50; 15.7%)P-value
      Chi-squared test for categorical variables; one-way ANOVA for continuous ones.
      Trial sample, median (IQR)419 (642)212 (260)137 (110)<0.001
      Location of the RCTs, %<0.001
       United States25.814.94.0
       Western Europe19.232.220.0
       Asia8.226.460.0
       International
      Trials conducted in countries from different continents.
      43.414.92.0
       Others3.311.514.0
      Context under evaluation, %<0.001
       Infectious diseases10.420.76.0
       Cardiovascular disorders21.417.26.0
       Cancer13.717.28.0
       Anesthesiology4.43.518.0
       Diabetes mellitus8.92.30.0
       Others41.239.162.0
      Type of intervention, n (%)0.8
       Drugs92.088.590.7
       Biologics8.011.59.3
      Compared interventions (ATC class), n (%)0.001
       Anti-infectives for systemic use9.914.98.0
       Antineoplastic and immunomodulating agents23.118.46.0
       Nervous system12.116.132.0
       Alimentary tract and metabolism13.217.216.0
       Cardiovascular system13.712.64.0
       Blood and blood forming organs11.54.64.0
       Genitourinary system and sex hormones3.33.514.0
       Antiparasitic products1.68.02.0
       Dermatologicals3.80.06.0
       Musculoskeletal system3.31.26.0
       Respiratory system3.31.20.0
       Sensory organs0.51.20.0
       Systemic hormonal preparations (excluding sex hormones)0.00.02.0
       Various0.51.10.0
      Trials with common backbone treatment, %25.831.022.00.5
      Registered trials, %86.373.62.0<0.001
      Trial design, %
      N = 318 (one trial with unspecified design was excluded from the comparison).
      0.012
       Superiority68.775.985.7
       Noninferiority29.118.48.2
       Equivalence2.25.76.1
      Trials with favorable results, %83.358.670.0<0.001
      Journal impact factor, median (IQR)4.0 (4.7)3.8 (4.0)1.3 (1.5)<0.001
      Number of citations, median (IQR)8 (23)5 (8)2.5 (4)<0.001
      Authors' affiliation, %<0.001
       Academic only14.859.876.0
       Both academic and nonprofit15.434.512.0
       Nonprofit only3.33.410.0
       Industry (at least one author)66.52.32.0
      Authors declaring a conflict of interest, %91.816.18.0<0.001
      Abbreviations: IQR, interquartile range; RCT, randomized controlled trial; ATC, Anatomical Therapeutic Chemical classification system.
      a Chi-squared test for categorical variables; one-way ANOVA for continuous ones.
      b Trials conducted in countries from different continents.
      c N = 318 (one trial with unspecified design was excluded from the comparison).

      3.4 Association of industry funding, sample size, and design with “positive” results

      In univariate analyses, industry funding, noninferiority/equivalence design, and larger sample size were associated with higher proportion of “favorable” results (Table 3). In multivariate analyses considering all three variables, although the prevalence of “favorable” results was significantly higher among industry trials than other trials [adjusted odds ratio (OR) 2.8; 95% confidence interval (CI): 1.6, 4.7; P < 0.001] and with noninferiority/equivalence design (adjusted OR 3.2; 95% CI: 1.5, 6.6; P = 0.002) compared with superiority design, it did not increase with larger sample size (Table 3). None of the other recorded variables was significantly associated with favorable findings at multivariate analysis. Almost all RCTs with a noninferiority/equivalence design funded by the industry had “favorable” findings (55 of 57, 96.5%).
      Table 3Relationship between favorable results, type of sponsorship, trial design and sample: percentage, odds ratio (OR) and 95% confidence interval (CI)


      Variables
      %
      Proportion or odds ratio of “favorable” results (see text for details).
      UnivariateMultivariate
      OR (95% CI)
      Proportion or odds ratio of “favorable” results (see text for details).
      P-valueOR (95% CI)
      Proportion or odds ratio of “favorable” results (see text for details).
      P-value
      Type of sponsorship:
       Not-for-profit institutions (reference category)62.811
       Pharmaceutical companies82.32.8 (1.7, 4.7)<0.0012.8 (1.6, 4.7)<0.001
      Trial design:
       Superiority (reference category)68.511
       Noninferiority/equivalence88.23.4 (1.7, 7.0)0.0013.2 (1.5, 6.6)0.002
      Sample size
       100–200 participants (reference category)69.611
       201–50075.01.3 (0.7, 2.3)0.40.9 (0.5, 1.8)0.9
       More than 50077.81.6 (0.8, 2.9)0.180.9 (0.5, 1.9)0.8
      a Proportion or odds ratio of “favorable” results (see text for details).

      4. Discussion

      Our empirical evaluation assessed 319 recent head-to-head randomized trials across a wide spectrum of treatments and conditions. Most head-to-head comparative evidence is procured by industry-sponsored trials. Typically, only one industry sponsor is involved, apparently with an objective to obtain evidence on its product that could be used for promotion. Noteworthy, the vast majority of industry trials were funded exclusively by companies, as only 2.8% were cosponsored by nonprofit institutions.
      Industry-sponsored trials tend to be larger, they are more frequently registered, and they tend to have higher citation impact and “favorable” results for the experimental treatment. Industry-sponsored trials also use noninferiority/equivalence designs more frequently than other trials. Industry funding and noninferiority/equivalence designs are strongly associated with “favorable” findings, and when they coexist, almost all trials get desirable “favorable” results.
      It has been speculated whether industry tends to avoid head-to-head comparisons to avoid jeopardizing its market share by unfavorable results [
      • Lathyris D.N.
      • Patsopoulos N.A.
      • Salanti G.
      • Ioannidis J.P.
      Industry sponsorship and selection of comparators in randomized clinical trials.
      ,
      • Dunn A.G.
      • Bourgeois F.T.
      • Murthy S.
      • Mandl K.D.
      • Day R.O.
      • Coiera E.
      The role and impact of research agendas on the comparative-effectiveness research among antihyperlipidemics.
      ,
      • Dunn A.G.
      • Mandl K.D.
      • Coiera E.
      • Bourgeois F.T.
      The effects of industry sponsorship on comparator selection in trial registrations for neuropsychiatric conditions in children.
      ]. However, we documented that among the head-to-head RCTs that are eventually performed, the industry still has the lion's share of this research agenda, which can be highly influential for informing guidelines and evidence-based practice. A preponderance of industry sponsorship in head-to-head trials has also been demonstrated in previous smaller evaluations of trials in specific areas of cardiovascular and psychiatric disorders [
      • Ridker P.M.
      • Torres J.
      Reported outcomes in major cardiovascular clinical trials funded by for-profit and not-for-profit organizations: 2000-2005.
      ,
      • Bero L.
      • Oostvogel F.
      • Bacchetti P.
      • Lee K.
      Factors associated with findings of published trials of drug-drug comparisons: why some statins appear more efficacious than others.
      ,
      • Heres S.
      • Davis J.
      • Maino K.
      • Jetzinger E.
      • Kissling W.
      • Leucht S.
      Why olanzapine beats risperidone, risperidone beats quetiapine, and quetiapine beats olanzapine: an exploratory analysis of head-to-head comparison studies of second-generation antipsychotics.
      ,
      • Gartlehner G.
      • Morgan L.
      • Thieda P.
      • Fleg A.
      The effect of study sponsorship on a systematically evaluated body of evidence of head-to-head trials was modest: secondary analysis of a systematic review.
      ] and H1N1 influenza vaccines [
      • Alsheikh-Ali A.A.
      • Qureshi W.
      • Al-Mallah M.H.
      • Ioannidis J.P.
      Public availability of published research data in high-impact journals.
      ,
      • Manzoli L.
      • De Vito C.
      • Salanti G.
      • D'Addario M.
      • Villari P.
      • Ioannidis J.P.
      Meta-analysis of the immunogenicity and tolerability of pandemic influenza A 2009 (H1N1) vaccines.
      ].
      We found that more than three-quarters of the head-to-head trials compared products owned by different companies, with no significant difference between industry and nonindustry-funded trials. However, we observed that cosponsorship of antagonistic trials was infrequent. Twenty-three RCTs were supported by two (or more) different companies; however, only in three of these (0.9% of the total sample) two sponsors, each owning an active treatment, were truly compared against each other. In the other RCTs, the cosponsoring companies shared the ownership of the same intervention(s), or one of the sponsors was the owner of the common backbone treatment given to all patients, or owned both the interventions—the standard of care and the new drug. In this way, sponsoring companies have to gain from the trial regardless of the outcome, unless a company owns and compares a new drug against an older one, which is about to go off patent. In this peculiar case, the company would not gain if the old drug proves to be as good, or even better, than the new one. Overall, each company tended to support almost exclusively trials focused on its own products, trying to prove its new agent superior or noninferior to some established comparator.
      As for randomized trials in general [
      • Okike K.
      • Kocher M.S.
      • Mehlman C.T.
      • Bhandari M.
      Industry-sponsored research.
      ], the profile and characteristics of head-to-head trials appear to be influenced by their sponsorship. Compared with nonindustry RCTs, industry trials had higher rates of registration, larger sample size, were published on journals with higher impact factors, and were more cited. All these findings tend to suggest that industry-sponsored head-to-head RCTs are outperforming other head-to-head trials. This is not surprising because supporting companies see these trials as an investment that is important for their marketing efforts, and thus, they want to optimize the compliance with requested policies (such as registration) and the impact of this research. Moreover, companies are expected to meet standard quality criteria during the planning and conducting of clinical trials and are subject to intense scrutiny by regulatory agencies [
      • Bhatt A.
      Quality of clinical trials: a moving target.
      ].
      Also, as previously hypothesized [
      • Heres S.
      • Davis J.
      • Maino K.
      • Jetzinger E.
      • Kissling W.
      • Leucht S.
      Why olanzapine beats risperidone, risperidone beats quetiapine, and quetiapine beats olanzapine: an exploratory analysis of head-to-head comparison studies of second-generation antipsychotics.
      ,
      • Garattini S.
      • Bertele V.
      Ethics in clinical research.
      ,
      • Apolone G.
      • Joppi R.
      • Bertele V.
      • Garattini S.
      Ten years of marketing approvals of anticancer drugs in Europe: regulatory policy and guidance documents need to find a balance between different pressures.
      ], we found that industry trials were more likely than other trials to adopt a noninferiority/equivalence design. There are two potential motivations for using noninferiority designs. First, often there can be advantages in terms of sample size (and, in turn, costs) [
      • Heres S.
      • Davis J.
      • Maino K.
      • Jetzinger E.
      • Kissling W.
      • Leucht S.
      Why olanzapine beats risperidone, risperidone beats quetiapine, and quetiapine beats olanzapine: an exploratory analysis of head-to-head comparison studies of second-generation antipsychotics.
      ,
      • Powers J.H.
      • Cooper C.K.
      • Lin D.
      • Ross D.B.
      Sample size and the ethics of non-inferiority trials.
      ]. Moreover, proving noninferiority of a new product may be sometimes less risky than aiming to establish its superiority because a finding of noninferiority will require less of a treatment effect yet still may be enough to support product approval [
      • Garattini S.
      • Bertele V.
      Ethics in clinical research.
      ,
      FDA
      Guidance for industry—non-inferiority clinical trials.
      ]. Along with this increasing trend, however, growing concerns occur: some argue that noninferiority RCTs mainly benefit companies, as they allow drugs without additional clinical efficacy to enter the market [
      • Garattini S.
      • Bertele V.
      Non-inferiority trials are unethical because they disregard patients' interests.
      ,
      • Gotzsche P.C.
      Lessons from and cautions about noninferiority and equivalence randomized trials.
      ,
      • Djulbegovic B.
      • Clarke M.
      Scientific and ethical issues in equivalence trials.
      ]. However, admittedly, these drugs may still be useful if they are safer and less expensive [
      • Chuang-Stein C.
      • Beltangady M.
      • Dunne M.
      • Morrison B.
      The ethics of non-inferiority trials.
      ,
      • Gandjour A.
      The ethics of non-inferiority trials.
      ,
      • Soliman E.Z.
      The ethics of non-inferiority trials.
      ]. We found that published industry-sponsored trials with noninferiority/equivalence designs were practically almost always successful to get a favorable, “positive” result, that is, claim (at least) noninferiority. Unfortunately, no information is available to evaluate whether such a high success rate occurs among all trials including those that remain unpublished or whether pharmaceutical companies selectively fund trials on drugs that they consider to be superior to the competition. In any case, by focusing on noninferiority, companies shift the traditional concept of equipoise where typically only about half of the trials manage to show superiority of the new treatment against a standard comparator [
      • Djulbegovic B.
      • Lacevic M.
      • Cantor A.
      • Fields K.K.
      • Bennett C.L.
      • Adams J.R.
      • et al.
      The uncertainty principle and industry-sponsored research.
      ,
      • Mann H.
      • Djulbegovic B.
      Choosing a control intervention for a randomised clinical trial.
      ,
      • Djulbegovic B.
      Acknowledgment of uncertainty: a fundamental means to ensure scientific and ethical validity in clinical research.
      ,
      • Mann H.
      • Djulbegovic B.
      • Gold P.
      Failure of equipoise to resolve the ethical tension in a randomized clinical trial.
      ,
      • Djulbegovic B.
      The paradox of equipoise: the principle that drives and limits therapeutic discoveries in clinical research.
      ] and select a design and inferential framework that is more likely to give them favorable results [
      • Gerlinger C.
      • Schmelter T.
      Determining the non-inferiority margin for patient reported outcomes.
      ].
      Although we found an independent, significant relationship between reporting positive findings and being sponsored by industry or having a noninferiority/equivalence design, trial results were not related to sample size. An association between industry sponsorship and positive findings has been documented by several surveys of clinical trials [
      • Ridker P.M.
      • Torres J.
      Reported outcomes in major cardiovascular clinical trials funded by for-profit and not-for-profit organizations: 2000-2005.
      ,
      • Bero L.
      • Oostvogel F.
      • Bacchetti P.
      • Lee K.
      Factors associated with findings of published trials of drug-drug comparisons: why some statins appear more efficacious than others.
      ,
      • Heres S.
      • Davis J.
      • Maino K.
      • Jetzinger E.
      • Kissling W.
      • Leucht S.
      Why olanzapine beats risperidone, risperidone beats quetiapine, and quetiapine beats olanzapine: an exploratory analysis of head-to-head comparison studies of second-generation antipsychotics.
      ,
      • Gartlehner G.
      • Morgan L.
      • Thieda P.
      • Fleg A.
      The effect of study sponsorship on a systematically evaluated body of evidence of head-to-head trials was modest: secondary analysis of a systematic review.
      ,
      • Lexchin J.
      • Bero L.A.
      • Djulbegovic B.
      • Clark O.
      Pharmaceutical industry sponsorship and research outcome and quality: systematic review.
      ,
      • Lundh A.
      • Sismondo S.
      • Lexchin J.
      • Busuioc O.A.
      • Bero L.
      Industry sponsorship and research outcome.
      ,
      • Bekelman J.E.
      • Li Y.
      • Gross C.P.
      Scope and impact of financial conflicts of interest in biomedical research: a systematic review.
      ,
      • Als-Nielsen B.
      • Chen W.
      • Gluud C.
      • Kjaergard L.L.
      Association of funding and conclusions in randomized drug trials: a reflection of treatment effect or adverse events?.
      ,
      • Manzoli L.
      • Flacco M.E.
      • D'Addario M.
      • Capasso L.
      • De Vito C.
      • Marzuillo C.
      • et al.
      Non-publication and delayed publication of randomized trials on vaccines: survey.
      ]. Sample size was previously found to be associated with favorable findings in few empirical surveys [
      • Bero L.
      • Oostvogel F.
      • Bacchetti P.
      • Lee K.
      Factors associated with findings of published trials of drug-drug comparisons: why some statins appear more efficacious than others.
      ,
      • Rasmussen N.
      • Lee K.
      • Bero L.
      Association of trial registration with the results and conclusions of published trials of new oncology drugs.
      ], only one of which focused on head-to-head trials [
      • Bero L.
      • Oostvogel F.
      • Bacchetti P.
      • Lee K.
      Factors associated with findings of published trials of drug-drug comparisons: why some statins appear more efficacious than others.
      ] and the association was seen for favorable trial conclusions rather than favorable trial results per se. Our analysis might also be partially affected by the exclusion of smaller RCTs.
      Previous literature has showed an association between the presence of author's conflict of interest and favorable findings in randomized trials [
      • Friedman L.S.
      • Richter E.D.
      Relationship between conflicts of interest and research results.
      ,
      • Kjaergard L.L.
      • Als-Nielsen B.
      Association between competing interests and authors' conclusions: epidemiological study of randomised clinical trials published in the BMJ.
      ,
      • Jagsi R.
      • Sheets N.
      • Jankovic A.
      • Motomura A.R.
      • Amarnath S.
      • Ubel P.A.
      Frequency, nature, effects, and correlates of conflicts of interest in published clinical cancer research.
      ]. Because of the strong correlation between funding source and conflict of interest (Spearman rho 0.77), the variable “conflict of interest” was not significant in multivariate analysis in our survey. However, if funding source was removed from the model, the presence of a conflict of interest would have been significantly associated with favorable findings (not shown).
      Some limitations should be acknowledged. First, it is possible that we failed to identify funding sources and financial ties in some of the included trials. We classified trials as industry funded or not based on each article's disclosure of its funding source(s), and only if no funding source was listed in the article, we checked the corresponding trial registration record, if available. Krimsky et al. [
      • Krimsky S.
      • Rothenberg L.S.
      Financial interest and its disclosure in scientific publications.
      ,
      • Krimsky S.
      • Rothenberg L.S.
      Conflict of interest policies in science and medical journals: editorial practices and author disclosures.
      ] showed, however, that there is a lack of disclosure of industry research support and personal financial ties across a wide variety of journals. Thus, we may have underestimated the number of industry-sponsored trials and personal financial ties of investigators. Second, we tried to ascertain all potential relationships between companies, but sometimes financial arrangements remain undetected in internet searches, especially those between small companies with larger ones [
      • Lathyris D.N.
      • Patsopoulos N.A.
      • Salanti G.
      • Ioannidis J.P.
      Industry sponsorship and selection of comparators in randomized clinical trials.
      ]. Third, because industry-funded trials and drug trials are more likely to have large samples [
      • Bero L.
      • Oostvogel F.
      • Bacchetti P.
      • Lee K.
      Factors associated with findings of published trials of drug-drug comparisons: why some statins appear more efficacious than others.
      ,
      • Smoak C.G.
      Regulatory submissions for medical devices and diagnostics: the basics.
      ], the exclusion from our analyses of RCTs with <100 patients may have led to underrepresentation of nonindustry-funded trials and device trials in our sample.
      Finally and most importantly, we only examined a (large) sample of RCTs published in 2011, and thus, we cannot map whether any changes in these patterns of head-to-head trials might have occurred over the years. However, our survey offers evidence on the profile of trials that were published in the recent literature.
      In conclusion, there is strong dominance of the industry in the influential agenda of head-to-head comparisons, confirming the unbalance between profit and nonprofit sponsored sources of data of current literature. We observed a high prevalence of results that were favorable for the sponsoring companies, which may have several explanations including: (1) industry trials may be conducted more rigorously than nonprofit trials and are thus genuinely more successful; (2) pharmaceutical companies may selectively fund trials that are more likely to yield favorable results (possibly due to the many preliminary phase 1 and phase 2 studies that are conducted before embarking on phase 3); (3) industry trials choose suboptimal outcomes, comparators, and other design features that can secure a favorable result; or (4) trials with unfavorable findings may be less likely to be published by companies. It is currently impossible to determine the relative weight of each of the above, but given the importance of head-to-head comparisons in informing guideline recommendations and practice, consideration should be given to allowing the conduct of more large trials of comparative effectiveness and safety under the control of nonprofit entities [
      • Ioannidis J.P.
      Mega-trials for blockbusters.
      ]. The design of such trials should be such as to inform important questions rather than pre-emptively ensure that results would be favorable for a tested intervention.

      Supplementary data

      References

        • Ioannidis J.P.
        Indirect comparisons: the mesh and mess of clinical trials.
        Lancet. 2006; 368: 1470-1472
        • Owens D.K.
        • Lohr K.N.
        • Atkins D.
        • Treadwell J.R.
        • Reston J.T.
        • Bass E.B.
        • et al.
        AHRQ series paper 5: grading the strength of a body of evidence when comparing medical interventions—agency for healthcare research and quality and the effective health-care program.
        J Clin Epidemiol. 2010; 63: 513-523
        • Zentner A.
        • Velasco-Garrido M.
        • Busse R.
        Methods for the comparative evaluation of pharmaceuticals.
        GMS Health Technol Assess. 2005; 1: Doc09
        • Estellat C.
        • Ravaud P.
        Lack of head-to-head trials and fair control arms: randomized controlled trials of biologic treatment for rheumatoid arthritis.
        Arch Intern Med. 2012; 172: 237-244
        • Ridker P.M.
        • Torres J.
        Reported outcomes in major cardiovascular clinical trials funded by for-profit and not-for-profit organizations: 2000-2005.
        JAMA. 2006; 295: 2270-2274
        • Bero L.
        • Oostvogel F.
        • Bacchetti P.
        • Lee K.
        Factors associated with findings of published trials of drug-drug comparisons: why some statins appear more efficacious than others.
        PLoS Med. 2007; 4: e184
        • Heres S.
        • Davis J.
        • Maino K.
        • Jetzinger E.
        • Kissling W.
        • Leucht S.
        Why olanzapine beats risperidone, risperidone beats quetiapine, and quetiapine beats olanzapine: an exploratory analysis of head-to-head comparison studies of second-generation antipsychotics.
        Am J Psychiatry. 2006; 163: 185-194
        • Gartlehner G.
        • Morgan L.
        • Thieda P.
        • Fleg A.
        The effect of study sponsorship on a systematically evaluated body of evidence of head-to-head trials was modest: secondary analysis of a systematic review.
        J Clin Epidemiol. 2010; 63: 117-125
        • Lathyris D.N.
        • Patsopoulos N.A.
        • Salanti G.
        • Ioannidis J.P.
        Industry sponsorship and selection of comparators in randomized clinical trials.
        Eur J Clin Invest. 2010; 40: 172-182
        • Sackett D.L.
        • Rosenberg W.M.
        • Gray J.A.
        • Haynes R.B.
        • Richardson W.S.
        Evidence based medicine: what it is and what it isn't.
        BMJ. 1996; 312: 71-72
        • Stamatakis E.
        • Weiler R.
        • Ioannidis J.P.
        Undue industry influences that distort healthcare research, strategy, expenditure and practice: a review.
        Eur J Clin Invest. 2013; 43: 469-475
        • DeAngelis C.D.
        The influence of money on medical science.
        JAMA. 2006; 296: 996-998
        • Patsopoulos N.A.
        • Ioannidis J.P.
        • Analatos A.A.
        Origin and funding of the most frequently cited papers in medicine: database analysis.
        BMJ. 2006; 332: 1061-1064
        • Melander H.
        • Ahlqvist-Rastad J.
        • Meijer G.
        • Beermann B.
        Evidence b(i)ased medicine—selective reporting from studies sponsored by pharmaceutical industry: review of studies in new drug applications.
        BMJ. 2003; 326: 1171-1173
        • Yank V.
        • Rennie D.
        • Bero L.A.
        Financial ties and concordance between results and conclusions in meta-analyses: retrospective cohort study.
        BMJ. 2007; 335: 1202-1205
        • Ioannidis J.P.
        • Greenland S.
        • Hlatky M.A.
        • Khoury M.J.
        • Macleod M.R.
        • Moher D.
        • et al.
        Increasing value and reducing waste in research design, conduct, and analysis.
        Lancet. 2014; 383: 166-175
        • Ross J.S.
        • Hill K.P.
        • Egilman D.S.
        • Krumholz H.M.
        Guest authorship and ghostwriting in publications related to rofecoxib: a case study of industry documents from rofecoxib litigation.
        JAMA. 2008; 299: 1800-1812
        • Hill K.P.
        • Ross J.S.
        • Egilman D.S.
        • Krumholz H.M.
        The ADVANTAGE seeding trial: a review of internal documents.
        Ann Intern Med. 2008; 149: 251-258
        • Lexchin J.
        • Bero L.A.
        • Djulbegovic B.
        • Clark O.
        Pharmaceutical industry sponsorship and research outcome and quality: systematic review.
        BMJ. 2003; 326: 1167-1170
        • Garattini S.
        • Bertele V.
        Non-inferiority trials are unethical because they disregard patients' interests.
        Lancet. 2007; 370: 1875-1877
        • Powers J.H.
        • Cooper C.K.
        • Lin D.
        • Ross D.B.
        Sample size and the ethics of non-inferiority trials.
        Lancet. 2005; 366: 24-25
        • Resnik D.B.
        • Peddada S.
        • Brunson Jr., W.
        Research misconduct policies of scientific journals.
        Account Res. 2009; 16: 254-267
        • Resnik D.B.
        • Patrone D.
        • Peddada S.
        Research misconduct policies of social science journals and impact factor.
        Account Res. 2010; 17: 79-84
        • Hopewell S.
        • Loudon K.
        • Clarke M.J.
        • Oxman A.D.
        • Dickersin K.
        Publication bias in clinical trials due to statistical significance or direction of trial results.
        Cochrane Database Syst Rev. 2009; : MR000006
        • Ioannidis J.P.
        Effect of the statistical significance of results on the time to completion and publication of randomized efficacy trials.
        JAMA. 1998; 279: 281-286
        • Lundh A.
        • Sismondo S.
        • Lexchin J.
        • Busuioc O.A.
        • Bero L.
        Industry sponsorship and research outcome.
        Cochrane Database Syst Rev. 2012; : MR000033
        • Alsheikh-Ali A.A.
        • Qureshi W.
        • Al-Mallah M.H.
        • Ioannidis J.P.
        Public availability of published research data in high-impact journals.
        PLoS One. 2011; 6: e24357
        • Dunn A.G.
        • Bourgeois F.T.
        • Murthy S.
        • Mandl K.D.
        • Day R.O.
        • Coiera E.
        The role and impact of research agendas on the comparative-effectiveness research among antihyperlipidemics.
        Clin Pharmacol Ther. 2012; 91: 685-691
        • Dunn A.G.
        • Mandl K.D.
        • Coiera E.
        • Bourgeois F.T.
        The effects of industry sponsorship on comparator selection in trial registrations for neuropsychiatric conditions in children.
        PLoS One. 2013; 8: e84951
        • Manzoli L.
        • De Vito C.
        • Salanti G.
        • D'Addario M.
        • Villari P.
        • Ioannidis J.P.
        Meta-analysis of the immunogenicity and tolerability of pandemic influenza A 2009 (H1N1) vaccines.
        PLoS One. 2011; 6: e24384
        • Okike K.
        • Kocher M.S.
        • Mehlman C.T.
        • Bhandari M.
        Industry-sponsored research.
        Injury. 2008; 39: 666-680
        • Bhatt A.
        Quality of clinical trials: a moving target.
        Perspect Clin Res. 2011; 2: 124-128
        • Garattini S.
        • Bertele V.
        Ethics in clinical research.
        J Hepatol. 2009; 51: 792-797
        • Apolone G.
        • Joppi R.
        • Bertele V.
        • Garattini S.
        Ten years of marketing approvals of anticancer drugs in Europe: regulatory policy and guidance documents need to find a balance between different pressures.
        Br J Cancer. 2005; 93: 504-509
        • FDA
        Guidance for industry—non-inferiority clinical trials.
        U.S. Food and Drug Administration, Silver Spring, MD, USA2010
        • Gotzsche P.C.
        Lessons from and cautions about noninferiority and equivalence randomized trials.
        JAMA. 2006; 295: 1172-1174
        • Djulbegovic B.
        • Clarke M.
        Scientific and ethical issues in equivalence trials.
        JAMA. 2001; 285: 1206-1208
        • Chuang-Stein C.
        • Beltangady M.
        • Dunne M.
        • Morrison B.
        The ethics of non-inferiority trials.
        Lancet. 2008; 371 (author reply 896–7): 895-896
        • Gandjour A.
        The ethics of non-inferiority trials.
        Lancet. 2008; 371 (author reply 896–7): 895
        • Soliman E.Z.
        The ethics of non-inferiority trials.
        Lancet. 2008; 371 (author reply 896–7): 895
        • Djulbegovic B.
        • Lacevic M.
        • Cantor A.
        • Fields K.K.
        • Bennett C.L.
        • Adams J.R.
        • et al.
        The uncertainty principle and industry-sponsored research.
        Lancet. 2000; 356: 635-638
        • Mann H.
        • Djulbegovic B.
        Choosing a control intervention for a randomised clinical trial.
        BMC Med Res Methodol. 2003; 3: 7
        • Djulbegovic B.
        Acknowledgment of uncertainty: a fundamental means to ensure scientific and ethical validity in clinical research.
        Curr Oncol Rep. 2001; 3: 389-395
        • Mann H.
        • Djulbegovic B.
        • Gold P.
        Failure of equipoise to resolve the ethical tension in a randomized clinical trial.
        J Law Med Ethics. 2003; 31: 5-6
        • Djulbegovic B.
        The paradox of equipoise: the principle that drives and limits therapeutic discoveries in clinical research.
        Cancer Control. 2009; 16: 342-347
        • Gerlinger C.
        • Schmelter T.
        Determining the non-inferiority margin for patient reported outcomes.
        Pharm Stat. 2011; 10: 410-413
        • Bekelman J.E.
        • Li Y.
        • Gross C.P.
        Scope and impact of financial conflicts of interest in biomedical research: a systematic review.
        JAMA. 2003; 289: 454-465
        • Als-Nielsen B.
        • Chen W.
        • Gluud C.
        • Kjaergard L.L.
        Association of funding and conclusions in randomized drug trials: a reflection of treatment effect or adverse events?.
        JAMA. 2003; 290: 921-928
        • Manzoli L.
        • Flacco M.E.
        • D'Addario M.
        • Capasso L.
        • De Vito C.
        • Marzuillo C.
        • et al.
        Non-publication and delayed publication of randomized trials on vaccines: survey.
        BMJ. 2014; 348: g3058
        • Rasmussen N.
        • Lee K.
        • Bero L.
        Association of trial registration with the results and conclusions of published trials of new oncology drugs.
        Trials. 2009; 10: 116
        • Friedman L.S.
        • Richter E.D.
        Relationship between conflicts of interest and research results.
        J Gen Intern Med. 2004; 19: 51-56
        • Kjaergard L.L.
        • Als-Nielsen B.
        Association between competing interests and authors' conclusions: epidemiological study of randomised clinical trials published in the BMJ.
        BMJ. 2002; 325: 249
        • Jagsi R.
        • Sheets N.
        • Jankovic A.
        • Motomura A.R.
        • Amarnath S.
        • Ubel P.A.
        Frequency, nature, effects, and correlates of conflicts of interest in published clinical cancer research.
        Cancer. 2009; 115: 2783-2791
        • Krimsky S.
        • Rothenberg L.S.
        Financial interest and its disclosure in scientific publications.
        JAMA. 1998; 280: 225-226
        • Krimsky S.
        • Rothenberg L.S.
        Conflict of interest policies in science and medical journals: editorial practices and author disclosures.
        Sci Eng Ethics. 2001; 7: 205-218
        • Smoak C.G.
        Regulatory submissions for medical devices and diagnostics: the basics.
        CDISC J. 2011; : 1-5
        • Ioannidis J.P.
        Mega-trials for blockbusters.
        JAMA. 2013; 309: 239-240

      Linked Article

      • Noninferiority is (too) common in noninferiority trials
        Journal of Clinical EpidemiologyVol. 71
        • Preview
          In their meta-analysis of 317 randomized trials that examined efficacy and safety of drugs, biologics, and medical devices in head-to-head comparisons, Flacco et al. [1] report that 97% (55 of 57) of industry-funded noninferiority/equivalence trials provided favorable results (defined as superiority or noninferiority of the experimental treatment) compared to 71% (20 of 28) of nonindustry-funded noninferiority/equivalence trials. We reviewed the data from our previously published meta-analysis of 175 noninferiority comparisons [2].
        • Full-Text
        • PDF