Advertisement
Original Article| Volume 143, P242-253, March 2022

Download started.

Ok

Clinical trials were missing from regulatory documents of extended-release methylphenidate for ADHD in adults: a case study of public documents

  • Kim Boesen
    Correspondence
    Corresponding author:
    Affiliations
    Meta-Research Innovation Center Berlin (METRIC-B), Berlin Institute of Health at Charité, QUEST Center for Responsible Research, Germany

    Nordic Cochrane Centre, Rigshospitalet, Copenhagen, Denmark
    Search for articles by this author
  • Karsten Juhl Jørgensen
    Affiliations
    Nordic Cochrane Centre, Rigshospitalet, Copenhagen, Denmark

    Centre for Evidence-Based Medicine Odense (CEBMO) and Cochrane Denmark, Department of Clinical Research, University of Southern Denmark, Odense, Denmark

    Open Patient data Exploratory Network (OPEN), Odense University Hospital, Odense, Denmark
    Search for articles by this author
  • Peter C Gøtzsche
    Affiliations
    Institute for Scientific Freedom, Copenhagen, Denmark
    Search for articles by this author
Open AccessPublished:November 06, 2021DOI:https://doi.org/10.1016/j.jclinepi.2021.10.027
Clinical trials were missing from regulatory documents of extended-release methylphenidate for ADHD in adults: a case study of public documents
Previous ArticleIncreased interaction and procedural flexibility favoured participation: Study across European cohorts of preterm born individuals
Next ArticleMethodology over metrics: Response to Van Calster et al

      Abstract

      Objectives

      To assess whether drug regulatory agencies decided on applications for extended-release methylphenidate for use in adult ADHD based on select samples of trials.

      Study design and setting

      Case series of publicly available regulatory documents. We matched an index of extended-release methylphenidate trials for adult ADHD with trials appearing in regulatory documents of extended-release methylphenidate applications. Trials and regulatory documents were identified as part of this systematic review (https://doi.org/10.1002/14651858.CD012857). We sought to identify missing trials in the regulatory documents and to clarify regulatory submission requirements.

      Results

      We indexed 18 trials and matched those with 13 drug applications (11 approved, 2 rejected) published by 7 agencies. There were trials missing in 7 (54%) of 13 applications, median 4 trials (range 1-6). The median proportion of missing trial participants was 45% (range 23% - 72%). Regulators seemingly require that all trials must be included in new drug applications, but wording is ambiguous.

      Conclusion

      In this sample of extended-release methylphenidate drug applications for adult ADHD, 7 of 13 regulatory decisions were missing entire trials according to public documents, even though regulatory requirements seem to stipulate that all available trials should be included in drug applications.

      Keywords

      What is new?
      What is already known on this topic
      • •
        Drug regulatory agencies base their decisions on clinical trials submitted by pharmaceutical companies.
      • •
        Little is known whether drug regulators are missing relevant clinical trials either sponsored by the applicants, publicly funded, or both, and thereby risk making decisions based on select samples of trials.
      What this study adds
      • •
        It was difficult to identify which trials drug regulatory agencies based their decisions on to approve or reject extended-release methylphenidate applications.
      • •
        According to publicly available documents, drug regulatory agencies were missing clinical trials in 7 (54%) of 13 regulatory decisions regarding extended-release methylphenidate for ADHD in adults.
      • •
        Current regulatory submission requirements stipulate that applicants must submit all available trials, but ambiguous wording make them open to interpretation.

      1. Background

      Selective publication of trials [
      • Dwan K
      • Altman DG
      • Arnaiz JA
      • Bloom J
      • Chan A-W
      • Cronin E
      • et al.
      Systematic review of the empirical evidence of study publication bias and outcome reporting bias.
      PLOS One. 2008; 3: e3081
      ,
      • Dwan K
      • Gamble C
      • Williamson PR
      • Kirkham JJ
      • the Reporting Bias Group
      Systematic review of the empirical evidence of study publication bias and outcome reporting bias — an updated review.
      PLoS ONE. 2013; : e66844
      ] and selective outcome reporting [
      • Chan AW
      • Hrobjartsson A
      • Haahr MT
      • Gøtzsche PC
      • Altman DG.
      Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles.
      JAMA. 2004; 291: 2457-2465
      ,
      • Goldacre B
      • Drysdale H
      • Dale A
      • Milosevic I
      • Slade E
      • Hartley P
      • et al.
      COMPare: a prospective cohort study correcting and monitoring 58 misreported trials in real time.
      Trials. 2019; 20: 118
      ] are common. For clinical trials published in medical journals, there can be important discrepancies compared to the raw trial data submitted to the drug regulators [
      • Jefferson T
      • Jones MA
      • Doshi P
      • Del Mar CB
      • Hama R
      • Thompson MJ
      • et al.
      Neuroaminidase inhibitors for preventing and treating influenza in adults and children.
      Cochrane Database Syst Rev. 2014; 4CD008965
      ,
      • Jørgensen L
      • Gøtzsche PC
      • Jefferson T.
      Benefits and harms of the human papillomavirus (HPV) vaccines: comparison of trial data from clinical study reports with corresponding trial register entries and journal publications.
      Syst Rev. 2020; 9: 42
      ,
      • Munkholm K
      • Paludan-Müller AS
      • Boesen K.
      Considering the methodological limitations in the evidence of antidepressants for depression: a reanalysis of a network meta-analysis.
      BMJ Open. 2019; 9e024886
      ] and patient reported outcomes and harms seem to be particularly underreported in published reports [
      • Wieseler B
      • Wolfram N
      • McGauran N
      • Kerekes MF
      • Vervölgyi V
      • Kohlepp P
      • et al.
      Completeness of reporting of patient-relevant clinical trial outcomes: comparison of unpublished clinical study reports with publicly available data.
      PLOS Medicine. 2013; 10e1001536
      ,
      • Noury JL
      • Nardo JM
      • Healy D
      • Jureidini J
      • Raven M
      • Tufanaru C
      • et al.
      Restoring Study 329: efficacy and harms of paroxetine and imipramine in treatment of major depression in adolescence.
      BMJ. 2015; 351: h4320
      ].
      In 2008, it was reported that 31% of trials of antidepressants included in new drug applications submitted to the US Food and Drug Administration (FDA) were never published in medical journals [
      • Turner EH
      • Matthews AM
      • Linardatos E
      • Tell RA
      • Rosenthal R.
      Selective publication of antidepressant trials and its influence on apparent efficacy.
      N Engl J Med. 2008; 358: 252-260
      ]. Most of these unpublished trials did not favour the antidepressant and were considered “negative” by the FDA [
      • Turner EH
      • Matthews AM
      • Linardatos E
      • Tell RA
      • Rosenthal R.
      Selective publication of antidepressant trials and its influence on apparent efficacy.
      N Engl J Med. 2008; 358: 252-260
      ]. The authors assumed that FDA's cohort of clinical trials was exhaustive and did not investigate whether the companies had conducted more clinical trials.
      When pharmaceutical companies apply for marketing authorisation of a new drug, they must submit an application in the form of a Common Technical Document [

      International Council for Harmonisation of technical requirements for pharmaceuticals for human use. organisation of the common technical document for the registration of pharmaceuticals for human use. M4. Current step 4 version. 15 June 2016. Available from: https://database.ich.org/sites/default/files/M4_R4__Guideline.pdf (accessed 5 Dec 2021).

      ]. This is the international standard for all major drug regulatory agencies, and the document must contain information regarding quality assurance, non-clinical studies, e.g. pharmacokinetic and toxicology studies, and clinical trial data in the form of clinical study reports [

      European Medicines Agency. ICH M4 Common technical document for the registration of pharmaceuticals for human use – efficacy. Revision 2. July 2016. Available from: https://www.ema.europa.eu/en/ich-m4e-common-technical-document-registration-pharmaceuticals-human-use-efficacy (accessed 5 Dec 2021).

      ,
      • Doshi P
      • Jefferson T.
      Clinical study reports of randomised controlled trials: an exploratory review of previously confidential industry reports.
      BMJ Open. 2013; 3e002496
      ].
      Comparative studies have found that clinical study reports of antidepressant trials obtained from the European Medicines Agency (EMA) [
      • Gøtzsche PC
      • Jørgensen AW.
      Opening up data at the European medicines agency.
      BMJ. 2011; 342: d2686
      ] and the UK regulator, Medicines and Healthcare products Regulatory Agency (MHRA), were incomplete [
      • Sharma T
      • Guski LS
      • Freund N
      • Gøtzsche PC.
      Suicidality and aggression during antidepressant treatment: systematic review and meta-analyses based on clinical study reports.
      BMJ. 2016; 352: i65
      ,
      • Paludan-Müller AS
      • Sharma T
      • Rasmussen K
      • Gøtzsche PC.
      Extensive selective reporting of quality of life in clinical study reports and publications of placebo-controlled trials of antidepressants.
      Int J Risk Saf Med. 2021; 32: 87-99
      ] and internally inconsistent [
      • Maund E
      • Tendal B
      • Hrobjartsson A
      • Jørgensen KJ
      • Lundh A
      • Schroll J
      • et al.
      Benefits and harms in clinical trials of duloxetine for treatment of major depressive disorder: comparison of clinical study reports, trial registries, and publications.
      BMJ. 2014; 348: g3510
      ]. In a systematic review [
      • Jefferson T
      • Jones MA
      • Doshi P
      • Del Mar CB
      • Hama R
      • Thompson MJ
      • et al.
      Neuroaminidase inhibitors for preventing and treating influenza in adults and children.
      Cochrane Database Syst Rev. 2014; 4CD008965
      ] of neuraminidase inhibitors for influenza based on clinical study reports, it was noted that the EMA and FDA “largely ignored” the largest oseltamivir trial, M76001, during the approval procedure (p. 28) [
      • Jefferson T
      • Jones MA
      • Doshi P
      • Del Mar CB
      • Hama R
      • Thompson MJ
      • et al.
      Neuroaminidase inhibitors for preventing and treating influenza in adults and children.
      Cochrane Database Syst Rev. 2014; 4CD008965
      ].
      To our knowledge, it has never been systematically assessed whether drug regulatory agencies may be missing entire clinical trials when they make decisions. Clinical study reports have generally not been easily accessible, but several drug regulatory agencies make documents publicly available about their authorised drugs [

      Restoring invisible and abandoned trials. Regulatory resources. Not dated. Available from: https://restoringtrials.org/regulatory-resources/ (accessed 5 Dec 2021).

      ].
      Based on a cohort of regulatory documents obtained for a systematic review [
      • Boesen K
      • Danborg PB
      • Gøtzsche PC
      • Jørgensen KJ.
      Extended-release methylphenidate for attention deficit hyperactivity disorder (ADHD) in adults (protocol).
      Cochrane Database Syst Rev. 2017; 11CD012857
      ,

      Boesen K, Paludan-Müller AS, Gøtzsche PC, Jørgensen KJ. Extended-release methylphenidate for attention deficit hyperactivity disorder (ADHD) in adults (review). Cochrane Database Syst Rev (In press).

      ] of extended-release methylphenidate for attention deficit hyperactivity disorder (ADHD) in adults, we wanted to systematically assess if drug regulatory agencies transparently report which trials they based their decisions on; are missing entire clinical trials prior to decision-making; and stipulate specific submission requirements that mandate the applicants to submit all trials in drug applications.

      2. Methods

      This is a case series based on publicly available drug regulatory documents. We obtained the documents during data collection for a systematic review on extended-release methylphenidate for adult ADHD [
      • Boesen K
      • Danborg PB
      • Gøtzsche PC
      • Jørgensen KJ.
      Extended-release methylphenidate for attention deficit hyperactivity disorder (ADHD) in adults (protocol).
      Cochrane Database Syst Rev. 2017; 11CD012857
      ,

      Boesen K, Paludan-Müller AS, Gøtzsche PC, Jørgensen KJ. Extended-release methylphenidate for attention deficit hyperactivity disorder (ADHD) in adults (review). Cochrane Database Syst Rev (In press).

      ]. Our systematic review was preregistered [
      • Boesen K
      • Danborg PB
      • Gøtzsche PC
      • Jørgensen KJ.
      Extended-release methylphenidate for attention deficit hyperactivity disorder (ADHD) in adults (protocol).
      Cochrane Database Syst Rev. 2017; 11CD012857
      ], but we did not publish a separate protocol for this analysis. Our results are reported according to the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) guideline [
      • von Elm E
      • Altman DG
      • Egger M
      • Pocock SJ
      • Gøtzsche PC
      • Vandenbroucke JP
      • et al.
      The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies.
      PLoS Med. 2007; 4: e296
      ]. See Supplement 1 for details on our methodology.

      2.1 Step 1: Index of clinical trials

      For our systematic review of extended-release methylphenidate for ADHD in adults [
      • Boesen K
      • Danborg PB
      • Gøtzsche PC
      • Jørgensen KJ.
      Extended-release methylphenidate for attention deficit hyperactivity disorder (ADHD) in adults (protocol).
      Cochrane Database Syst Rev. 2017; 11CD012857
      ,

      Boesen K, Paludan-Müller AS, Gøtzsche PC, Jørgensen KJ. Extended-release methylphenidate for attention deficit hyperactivity disorder (ADHD) in adults (review). Cochrane Database Syst Rev (In press).

      ], two authors (KB and ASP) systematically searched databases and trial registries (October 2017, March 2019, February 2021). In addition, one author (KB) searched pharmaceutical databases (May 2020), drug regulatory agency databases (May 2020, Aug 2021), and corresponded with trial authors (continuously from 2016 to 2021). Using this data, we created an index of randomised, placebo-controlled, extended-release methylphenidate clinical trials for ADHD in adults. For this analysis, we were interested in clinical trials sponsored by pharmaceutical companies or publicly funded trials with industry-involvement. We defined ‘industry-involvement’ as any type of collaboration and/or sponsorship with the pharmaceutical companies that market the relevant methylphenidate formulation, as stated in acknowledgement sections or declared in trial registries. Purely publicly funded trials rarely form the basis for new drug applications.

      2.2 Step 2: Drug regulatory documents

      One author (KB) manually searched (May 2020, Aug 2021) drug regulatory databases of publicly available documents: The FDA's database of Drug Approval Packages, the Australian Therapeutic Goods Administration's (TGA) AusPAR Database and Register of Therapeutic Goods, Health Canada's Drug Product Database, EMA's database of Public Assessment Reports, the Japanese Pharmaceuticals and Medical Devices Agency (PMDA) database of drug reports and approved drugs, and national European drug regulatory agency databases. See Supplement 1 and our systematic review for details [
      • Boesen K
      • Danborg PB
      • Gøtzsche PC
      • Jørgensen KJ.
      Extended-release methylphenidate for attention deficit hyperactivity disorder (ADHD) in adults (protocol).
      Cochrane Database Syst Rev. 2017; 11CD012857
      ,

      Boesen K, Paludan-Müller AS, Gøtzsche PC, Jørgensen KJ. Extended-release methylphenidate for attention deficit hyperactivity disorder (ADHD) in adults (review). Cochrane Database Syst Rev (In press).

      ].
      One author (KB) assessed each regulatory document and extracted information on all clinical trials appearing in the documents. They were categorised as ‘pivotal trials’, i.e., the main trials used for approval, or ‘other trials’, i.e., any other (identifiable) trial.

      2.3 Step 3: Matching index trials with regulatory documents

      We matched the index trials with those trials identified in the regulatory documents. We categorised any index trial not appearing in the regulatory documents as ‘missing’ if the trial was: (1) completed or ongoing at the time of decision-making, and (2) the trial was sponsored by the pharmaceutical company that had applied for drug approval, or (3) it was a publicly funded trial with ‘industry-involvement’ according to our definition.

      2.4 Step 4: Enquiring with regulators on missing trials

      For the applications with missing trials, we enquired with the regulators whether the applicants had submitted information on more trials than those appearing from the public documents. We did not enquire about applications with no missing trials.

      2.5 Step 5: Clarifying regulatory requirements for new drug applications

      One author (KB) searched the FDA, the TGA, Health Canada, and the EMA websites (April 2020) for guidelines and legislations regarding stipulated submission requirements of available clinical trial evidence. We did not assess individual European drug regulatory agency requirements, since clinical trial conduct, reporting, and marketing authorisation will be harmonised in the European Union, once the EU Clinical Trial Regulation is applied in the beginning of 2022 [
      Regulation No. 536/2014 of the European Parliament and of the Council
      Official J Euro Union. 2014; (27 May 2014. Available from:) (accessed 5 Dec 2021): L158/1
      https://ec.europa.eu/health/sites/default/files/files/eudralex/vol-1/reg_2014_536/reg_2014_536_en.pdf
      ,

      European Medicines Agency. Clinical Trial Regulation. Not dated. Available from: https://www.ema.europa.eu/en/human-regulatory/research-development/clinical-trials/clinical-trial-regulation (accessed 5 Dec 2021).

      ]. We did not search PMDA's website as it is mainly in Japanese. During our systematic review [
      • Boesen K
      • Danborg PB
      • Gøtzsche PC
      • Jørgensen KJ.
      Extended-release methylphenidate for attention deficit hyperactivity disorder (ADHD) in adults (protocol).
      Cochrane Database Syst Rev. 2017; 11CD012857
      ,

      Boesen K, Paludan-Müller AS, Gøtzsche PC, Jørgensen KJ. Extended-release methylphenidate for attention deficit hyperactivity disorder (ADHD) in adults (review). Cochrane Database Syst Rev (In press).

      ], we asked five regulatory agencies (FDA, TGA, Health Canada, MHRA, and BfArM) to clarify uncertainties in the publicly available documents and to specify whether submission of data from all existing trials, regardless of sponsorship, results, and trial status, was required for the drug approval process. During preparation of this manuscript, we additionally asked Health Canada, MHRA, and PMDA to clarify uncertainties related to three applications and we asked EMA about their submission requirements.

      3. Results

      3.1 Index of clinical trials

      We identified 16 industry-sponsored trials and 2 publicly funded trials with ‘industry involvement’, Table 1. Six different extended-release methylphenidate formulations were tested: Osmotic-controlled release oral delivery system (OROS) methylphenidate (Janssen, Concerta; 10 trials), extended-release (ER) methylphenidate (Medice, Medikinet;three trials), controlled-release (CR) methylphenidate (Purdue, Foquest/Adhansia;two trials), controlled-release (CR) methylphenidate (Purdue, Biphentin;one trial), long-acting (LA) methylphenidate (Novartis, Ritalin LA/XL;one trial), and dex-methylphenidate extended-release (Novartis, Focalin XR;one trial). All trials assessed benefits and harms. 12 trials had similar trial characteristics and six trials had particular design characteristics: Reimherr 2004 and Jain 2003 used a cross-over design; Chronis-Tuscano 2004 included women with ADHD who also had daughters with ADHD; Winhusen 2005 (publicly-funded but with industry involvement) also assessed smoking-cessation related outcomes; COMPAS 2007 (publicly-funded but with industry involvement) used a factorial design to also assess the impact of clinical management or group psychotherapy; and Wigal 2014 used a 'simulated work environment' in a cross-over trial design. See key trial characteristics in Supplement 1, eTable 2 .
      Table 1Index of extended-release methylphenidate trials in adult ADHD.
      TrialClinical trial registryFormulationStudy start -stopSponsorIndustry-involvement
      Biederman 2003NCT00181571OROS methylphenidateJune 2003 – Aug 2007McNeil and Janssen
      On ClinicalTrials.gov, Massachusetts General Hospital was listed as sponsor and McNeil as ‘collaborator’. In published reports (Biederman et al. 2006, 2010, and 2011), McNeil and Janssen were listed as sponsors.
      		Industry trial
      Reimherr 2004Not registeredOROS methylphenidateAug 2004 – Dec 2005McNeil
      According to the published report, it was ‘funded in part’ by McNeil.
      		Industry trial
      Chronis-Tuscano 2004NCT00318981OROS methylphenidateDec 2004 – Dec 2006McNeil
      On ClinicalTrials.gov, University of Maryland was listed as sponsor and McNeil as collaborator. In the published report (Chronis-Tuscano et al, 2008) McNeil was listed as sponsor.
      		Industry trial
      Medori 2005NCT00246220OROS methylphenidateMar 2005 – Nov 2006JanssenIndustry trial
      Winhusen 2005NCT00253747OROS methylphenidateNov 2005 – Mar 2008University of CincinnatiOrtho-McNeil Janssen listed as collaborator
      According to ClinicalTrials.gov.
      Adler 2006NCT00326391OROS methylphenidateApr 2006 – Dec 2006JanssenIndustry trial
      Casas 2008NCT00714688OROS methylphenidateFeb 2008 – Apr 2009JanssenIndustry trial
      Weisler 2009NCT00880217OROS methylphenidateMay 2009 – Jan 2010JanssenIndustry trial
      Goodman 2009NCT00937040OROS methylphenidateJuly 2009 – Feb 2010JanssenIndustry trial
      Takahashi 2011NCT01323192OROS methylphenidateMarch 2011 – April 2012JanssenIndustry trial
      Spencer 2003Not registeredDex-methylphenidate extended-releaseApr 2003 – Sep 2003NovartisIndustry trial
      Huss 2010NCT01259492Long-acting methylphenidateNov 2010 – Aug 2012NovartisIndustry trial
      Rösler 2004NCT00619840Extended-release methylphenidateNov 2004 – May 2006MediceIndustry trial
      Compas 2007

      		EUCTR 2006-000222-31Extended-release methylphenidateApril 2007 – Aug 2011German Ministry of Education and ResearchMedice was involved in the trial design and data collection
      According to the published report [43].
      Retz 2008NCT00730249Extended-release methylphenidateSep 2008 – Jan 2010MediceIndustry trial
      Jain 2003Not registeredControlled-release methylphenidateOct 2003 – April 2004PurdueIndustry trial
      Wigal 2014NCT02225639Controlled release methylphenidateAug 2014 – May 2015PurdueIndustry trial
      Weiss 2014NCT02139124Controlled-release methylphenidateOct 2014 – Jan 2015PurdueIndustry trial
      i On ClinicalTrials.gov, Massachusetts General Hospital was listed as sponsor and McNeil as ‘collaborator’. In published reports (Biederman et al. 2006, 2010, and 2011), McNeil and Janssen were listed as sponsors.
      ii According to the published report, it was ‘funded in part’ by McNeil.
      iii On ClinicalTrials.gov, University of Maryland was listed as sponsor and McNeil as collaborator. In the published report (Chronis-Tuscano et al, 2008) McNeil was listed as sponsor.
      iv According to ClinicalTrials.gov.
      v According to the published report

      NCT00730249. Quality Assurance of Administering Methylphenidate in Adults With Attention Deficit Hyperactivity Disorder (ADHD) - QUMEA (QUMEA). Last updated: 13 January 2010. Available from: https://clinicaltrials.gov/ct2/show/NCT00730249 (accessed 5 Dec 2021).

      Table 2Publicly available regulatory documents.
      eCaseRegulatorApplicationApplicantRegulatory documentsDecision
      1Health CanadaOROS methylphenidateJanssenProduct Monograph

      Health Canada. Concerta Product Monograph. Submission control no. 224296. 17 April 2019. Available from: https://pdf.hres.ca/dpd_pm/00050777.PDF (accessed 5 Dec 2021).

      		Approved on 15 April 2008
      2FDAOROS methylphenidateJanssenDrug Approval Package

      Center for Drug Evaluation and Research. Drug Approval Package. 21-121/S-017. 27 June 2008. Available from: https://www.accessdata.fda.gov/drugsatfda_docs/nda/2008/021121s017.pdf (accessed 5 Dec 2021).

      		Approved on 27 June 2008
      3TGAOROS methylphenidateJanssenProduct Information

      Therapeutic Goods Administration. Concerta extended release tables. CCDS181023. Date of revision 21 Oct 2021. Available from: https://www.ebs.tga.gov.au/ebs/picmi/picmirepository.nsf/pdf?OpenAgent&id=CP-2009-PI-00919-3&d=202005111016933 (accessed 5 Dec 2021).

      		Approved on 21 Jan 2009
      4MHRAOROS methylphenidateJanssenPublic Assessment Report

      Medicines and Healthcare Products Regulatory Agency. Public Assessment Report. Concerta XL 27 mg prolonged-release tablets. UK/H/0544/004/MR. Available from: https://mhraproductsproduction.blob.core.windows.net/docs/e16cca8676c72aff9c51ed19f93314a251e61a05 (accessed 5 Dec 2021).

      		Rejected on 14 July 2010
      5PMDAOROS methylphenidateJanssen‘Summary of Application’ report

      Pharmaceuticals and Medical Devices Agency. Summary of application materials. 17 January 2014. Available from: https://www.pmda.go.jp/drugs/2013/P201300158/index.html (accessed 5 Dec 2021).

      		Approved on 20 Dec 2013
      6BfArMExtended-release methylphenidateMedicePublic Assessment Report

      Bundesinstitut für Arzneimittel und Medizinprodukte. Public Assessment Report Summary [Öffentlicher Bewertungsbericht]. Medikinet Adult. 18 July 2011. Available from: https://portal.dimdi.de/amispb/doc/2011/07/25/2163890/OBFMBD5B836E01CC4AAA.rtf (accessed 5 Dec 2021).

      		Approved on 18 July 2011
      7FDADex-methylphenidate extended-releaseNovartisDrug Approval Package

      Center for Drug Evaluation and Research. Drug Approval Package. Focalin XR. 26 May 2005. Available from: https://www.accessdata.fda.gov/drugsatfda_docs/nda/2005/021802s000TOC.cfm (accessed 5 Dec 2021 ).

      		Approved on 26 May 2005
      8MEBLong-acting methylphenidateNovartisPublic Assessment Report
      Medicines Evaluation Board. Public Assessment Report Ritalin LA
      NL License RVG. 2021; 116377 (20 October 2016. Available from:) (accessed 5 Dec 2021): 116379-116382
      https://www.geneesmiddeleninformatiebank.nl/Pars/h116382.pdf
      		Rejected on 20 Oct 2016
      9BfArM
      BfArM approved the drug but MHRA published the Public Assessment Report.
      		Long-acting methylphenidateNovartisPublic Assessment Report
      Medicines and Healthcare products Regulatory Agency
      Public Assessment Report. Ritalin XL. UK License. 2018; (. No: 00101/0973-0977. March 2018. Available from: https://mhraproductsproduction.blob.core.windows.net/docs/4781baf366b1fafd0ea203962ccb54faadcdcfcc (accessed 5 Dec 2021))
      		Approved in 2017
      10TGALong-acting methylphenidateNovartisProduct Information

      Therapeutic Goods Administration. Australian Product Information. Ritalin 10/Ritalin LA. Latest amendment 14 Aug 2020. Available from: https://www.ebs.tga.gov.au/ebs/picmi/picmirepository.nsf/pdf?OpenAgent&id=CP-2010-PI-03175-3 (accessed 5 Dec 2021).

      		Approved in 2014
      11Health CanadaControlled-release methylphenidatePurdueProduct Monograph

      Health Canada. Product Monograph. Foquest. Submission Control No: 214860. 1 March 2019. Available from: https://pdf.hres.ca/dpd_pm/00049955.PDF (accessed 5 Dec 2021).

      ,

      Clinical Information Package

      Health Canada. Clinical Information. Foquest – submission control no. 187330. Release date 29 May 2020. Available from: https://clinical-information.canada.ca/ci-rc/item/187330. [Accessed 5 Dec 2021].

      		Approved on 14 Dec 2017
      12FDAControlled-release methylphenidatePurdueDrug Approval Package

      Center for Drug Evaluation and Research. Drug Approval Package. Adhansia XR. 27 Feb 2019. Available from: https://www.accessdata.fda.gov/drugsatfda_docs/nda/2019/212038Orig1s000TOC.cfm (accessed 5 Dec 2021).

      		Approved on 27 Feb 2019
      13Health CanadaControlled-release methylphenidatePurdueProduct Monograph

      Health Canada. Product Monograph. Biphentin. Revision 14 January 2020. Available from: https://pdf.hres.ca/dpd_pm/00054671.PDF (accessed 5 Dec 2021).

      		Approved (unknown date)
      i BfArM approved the drug but MHRA published the Public Assessment Report.

      3.2 Public drug regulatory documents

      We included drug regulatory documents related to 13 drug applications [

      Health Canada. Concerta Product Monograph. Submission control no. 224296. 17 April 2019. Available from: https://pdf.hres.ca/dpd_pm/00050777.PDF (accessed 5 Dec 2021).

      ,

      Center for Drug Evaluation and Research. Drug Approval Package. 21-121/S-017. 27 June 2008. Available from: https://www.accessdata.fda.gov/drugsatfda_docs/nda/2008/021121s017.pdf (accessed 5 Dec 2021).

      ,

      Therapeutic Goods Administration. Concerta extended release tables. CCDS181023. Date of revision 21 Oct 2021. Available from: https://www.ebs.tga.gov.au/ebs/picmi/picmirepository.nsf/pdf?OpenAgent&id=CP-2009-PI-00919-3&d=202005111016933 (accessed 5 Dec 2021).

      ,

      Medicines and Healthcare Products Regulatory Agency. Public Assessment Report. Concerta XL 27 mg prolonged-release tablets. UK/H/0544/004/MR. Available from: https://mhraproductsproduction.blob.core.windows.net/docs/e16cca8676c72aff9c51ed19f93314a251e61a05 (accessed 5 Dec 2021).

      ,

      Pharmaceuticals and Medical Devices Agency. Summary of application materials. 17 January 2014. Available from: https://www.pmda.go.jp/drugs/2013/P201300158/index.html (accessed 5 Dec 2021).

      ,

      Bundesinstitut für Arzneimittel und Medizinprodukte. Public Assessment Report Summary [Öffentlicher Bewertungsbericht]. Medikinet Adult. 18 July 2011. Available from: https://portal.dimdi.de/amispb/doc/2011/07/25/2163890/OBFMBD5B836E01CC4AAA.rtf (accessed 5 Dec 2021).

      ,

      Center for Drug Evaluation and Research. Drug Approval Package. Focalin XR. 26 May 2005. Available from: https://www.accessdata.fda.gov/drugsatfda_docs/nda/2005/021802s000TOC.cfm (accessed 5 Dec 2021 ).

      ,
      Medicines Evaluation Board. Public Assessment Report Ritalin LA
      NL License RVG. 2021; 116377 (20 October 2016. Available from:) (accessed 5 Dec 2021): 116379-116382
      https://www.geneesmiddeleninformatiebank.nl/Pars/h116382.pdf
      ,
      Medicines and Healthcare products Regulatory Agency
      Public Assessment Report. Ritalin XL. UK License. 2018; (. No: 00101/0973-0977. March 2018. Available from: https://mhraproductsproduction.blob.core.windows.net/docs/4781baf366b1fafd0ea203962ccb54faadcdcfcc (accessed 5 Dec 2021))
      ,

      Therapeutic Goods Administration. Australian Product Information. Ritalin 10/Ritalin LA. Latest amendment 14 Aug 2020. Available from: https://www.ebs.tga.gov.au/ebs/picmi/picmirepository.nsf/pdf?OpenAgent&id=CP-2010-PI-03175-3 (accessed 5 Dec 2021).

      ,

      Health Canada. Product Monograph. Foquest. Submission Control No: 214860. 1 March 2019. Available from: https://pdf.hres.ca/dpd_pm/00049955.PDF (accessed 5 Dec 2021).

      ,

      Health Canada. Clinical Information. Foquest – submission control no. 187330. Release date 29 May 2020. Available from: https://clinical-information.canada.ca/ci-rc/item/187330. [Accessed 5 Dec 2021].

      ,

      Center for Drug Evaluation and Research. Drug Approval Package. Adhansia XR. 27 Feb 2019. Available from: https://www.accessdata.fda.gov/drugsatfda_docs/nda/2019/212038Orig1s000TOC.cfm (accessed 5 Dec 2021).

      ,

      Health Canada. Product Monograph. Biphentin. Revision 14 January 2020. Available from: https://pdf.hres.ca/dpd_pm/00054671.PDF (accessed 5 Dec 2021).

      ], Table 2. They were published by seven different agencies: The FDA (three applications), Health Canada (three), the TGA (two), MHRA (two), PMDA (one), the Dutch Medicines Evaluation Board (MEB) (one), and the German Bundesinstitut für Arzneimittel und Medizinprodukte (BfArM) (one). 11 applications were approved and two were rejected, OROS methylphenidate by MHRA [

      Medicines and Healthcare Products Regulatory Agency. Public Assessment Report. Concerta XL 27 mg prolonged-release tablets. UK/H/0544/004/MR. Available from: https://mhraproductsproduction.blob.core.windows.net/docs/e16cca8676c72aff9c51ed19f93314a251e61a05 (accessed 5 Dec 2021).

      ] and long-acting methylphenidate by MEB [
      Medicines Evaluation Board. Public Assessment Report Ritalin LA
      NL License RVG. 2021; 116377 (20 October 2016. Available from:) (accessed 5 Dec 2021): 116379-116382
      https://www.geneesmiddeleninformatiebank.nl/Pars/h116382.pdf
      ].
      The documents varied in length and content. FDA's Approval Packages reported detailed results and summaries of the scientific discussions and Health Canada's Clinical Information Package included parts of the clinical study reports, whereas Health Canada and the TGA's prescriber documents reported sparse clinical trial data and did not include scientific discussions. The European drug regulators’ Public Assessment Reports varied from BfArM's 7-page report [

      Center for Drug Evaluation and Research. Drug Approval Package. Focalin XR. 26 May 2005. Available from: https://www.accessdata.fda.gov/drugsatfda_docs/nda/2005/021802s000TOC.cfm (accessed 5 Dec 2021 ).

      ] on extended-release methylphenidate to MHRA's 130-page report [

      Medicines and Healthcare Products Regulatory Agency. Public Assessment Report. Concerta XL 27 mg prolonged-release tablets. UK/H/0544/004/MR. Available from: https://mhraproductsproduction.blob.core.windows.net/docs/e16cca8676c72aff9c51ed19f93314a251e61a05 (accessed 5 Dec 2021).

      ] on OROS methylphenidate. We describe key document characteristics in Supplement 1, eTable 3 .
      Table 3Included and missing participants in regulatory documents.
      eCaseRegulatory agencyApplicationIncluded participants (%)Missing participants (%)Total participants
      Ongoing trials at time of regulatory decision-making were included with their final sample size.
      1Health CanadaOROS methylphenidate402 (28%)1057 (72%)1459
      2FDAOROS methylphenidate631 (44%)828 (56%)1459
      5PMDAOROS methylphenidate1194 (53%)1048 (47%)2242
      6BfArMExtended-release methylphenidate521 (55%)433 (45%)954
      4MHRAOROS methylphenidate1204 (61%)754 (39%)1958
      3TGAOROS methylphenidate910 (63%)549 (37%)1459
      10TGALong-acting methylphenidate725 (77%)221 (23%)946
      8MEBLong-acting methylphenidate946 (100%)0946
      9BfArMLong-acting methylphenidate946 (100%)0946
      11Health CanadaControlled-release methylphenidate
      The formulation Foquest/Adhansia
      		435 (100%)0435
      12FDAControlled-release methylphenidate
      The formulation Foquest/Adhansia
      		435 (100%)0435
      7FDADex-methylphenidate extended-release221 (100%)0221
      13Health CanadaControlled-release methylphenidate
      The formulation Biphentin
      		50 (100%)050
      i Ongoing trials at time of regulatory decision-making were included with their final sample size.
      ii The formulation Foquest/Adhansia
      iii The formulation Biphentin

      3.3 Matching index trials with regulatory documents

      We identified missing trials in 7 (54%) of the 13 drug applications. The median number of missing trials was 4 (range 1 to 6). In Fig. 1, we depict the included and missing trials and they are tabulated in Supplement 1, eTable 4 . The median proportion of missing trial participants of the total population was 45%, ranging from TGA's approval of long-acting methylphenidate (23%) to Health Canada's approval of OROS methylphenidate (72%), Table 3.
      Fig 1
      Fig. 1Included and missing trials according to publicly available documents. Numbers correspond to the 13 drug applications (see also Supplement 1, eTable 4).
      A= Adler 2006, B= Biederman, 2003, C= Casas 2008, E= Weisler 2009, G= Goodman 2009, H= Huss 2010, I= Weiss 2014, J= Jain 2003, L= Wigal 2014, M= Medori 2005, N= Chronis-Tuscano 2004, O= COMPAS 2007, Ö=Rösler 2004, R= Reimherr 2004, S= Spencer 2003, T= Takahashi 2011, W= Winhusen 2005, Z= Retz 2008.
      Table 4Regulatory submission requirements.
      Regulatory agencySubmission requirements
      FDAFDA's guideline on New Drug Applications

      Center for Drug Evaluation and Research. Guideline for the Format and Content of the Clinical and Statistical Sections of an Application. July 1988. Available from: https://www.fda.gov/media/71436/download (accessed 5 Dec 2021).



      “All controlled clinical studies, including incomplete or abandoned studies, and all pertinent data, whether developed with support of the sponsor or obtained from any other source should be presented in this section” (section D, 1, c)

      US Electronic Code of Federal Regulation

      Electronic Code of Federal Regulation. Title 21, chapter 1, subchapter d, part 314, subpart B, §314.50. Updated 2 April 2020. Available from: https://www.ecfr.gov/cgi-bin/text-idx?SID=6d2c00e689dbe39f96cc4fe77ce33908&mc=true&node=se21.5.314_150&rgn=div8 (accessed 5 Dec 2021).



      “A description and analysis of each controlled clinical study pertinent to a proposed use of the drug, including the protocol and a description of the statistical analyses used to evaluate the study” (Title 21, part 314)
      Health Canada

      		Submission Certificate for a New Drug Submission

      Health Canada. Submission certificate for a NDS, SNDS, SANDS, ANDS, or NC. In effect 1 April 2020. Available from: https://www.canada.ca/content/dam/hc-sc/documents/services/drugs-health-products/drug-products/applications-submissions/forms/certnds_attpdn-2020-eng.doc (accessed 5 Dec 2021).



      “[…] a) all pivotal studies included in the submission are complete and final comprehensive analyses provided;

      b) all pivotal data necessary to support the proposed indications, doses and formulations have been provided […]”
      EMA and TGA
      TGA follows the EMA standards [52,53].
      		EMA's guideline on the Common Technical Document

      European Medicines Agency. ICH M4 Common Technical Document for the registration of pharmaceuticals for human use – efficacy. Revision 2. July 2016. Available from: https://www.ema.europa.eu/en/ich-m4e-common-technical-document-registration-pharmaceuticals-human-use-efficacy (accessed 5 Dec 2021).



      “The purpose of this section is to present a critical analysis of the clinical data pertinent to the efficacy of the medicinal product in the intended population. The analysis should consider all relevant data, whether positive or negative […]. Those studies deemed relevant for evaluation of efficacy should be identified, and reasons that any apparently adequate and well-controlled studies are not considered relevant should be provided. Prematurely terminated studies should be noted and their impact considered”(section 2.5.4)

      EU Clinical Trial Directive (2001/83/EC)

      European Commission. Directive 2001/83/EC. 6 November 2001. Available from: https://ec.europa.eu/health//sites/health/files/files/eudralex/vol-1/dir_2001_83_consol_2012/dir_2001_83_cons_2012_en.pdf (accessed 5 Dec 2021).



      “Consequently, an essential requirement is that the results of all clinical trials should be communicated, both favourable and unfavourable” (Annex I, section 5.2)
      i TGA follows the EMA standards

      Therapeutic Goods Administration. Mandatory requirements for an effective application. Feb 2018. Available from: https://www.tga.gov.au/publication/mandatory-requirements-effective-application (accessed 5 Dec 2021).

      ,

      Therapeutic Goods Administration. International scientific guidelines adopted in Australia. 5 Aug 2020. Available from: https://www.tga.gov.au/ws-sg-index (accessed 5 Dec 2021).

      .
      In Box 1, we illustrate the potential impact of missing trials (Cases 1 and 2), and how a pharmaceutical company was involved in a publicly funded trial, which was not assessed in the initial application (Case 3). We describe all drug applications in Supplement 1, eCase 1 to eCase 13.

      3.4 Inquiring with regulators on missing trials

      We contacted the responsible agencies, i.e., FDA, TGA, Health Canada, BfArM, and MHRA, for the applications with missing trials (Supplement 1 eTable 5 and Supplement 2). We asked them whether they had received information on more trials than those appearing from the public documents. We did not contact PMDA, as the tabulated lists in their ‘Summary of Application’ document [

      Pharmaceuticals and Medical Devices Agency. Summary of application materials. 17 January 2014. Available from: https://www.pmda.go.jp/drugs/2013/P201300158/index.html (accessed 5 Dec 2021).

      ] resembled Clinical Study Report tables and seemed exhaustive.
      The FDA, TGA, and Health Canada abstained from directly addressing our questions on the missing trials, whereas BfArM confirmed that they had not assessed the missing COMPAS 2007 trial. The FDA referred us to their Drug Approval Database website and Health Canada sent us Module 2.7 from the OROS methylphenidate clinical study report, which contains a list of all clinical trials submitted as part of the application. After our manuscript was accepted for publication, Health Canada responded to an inquiry about controlled-release methylphenidate. Health Canada released a Clinical Information Package after we had searched their database in May 2020. The disclosed clinical study report documents unveiled a discrepancy with other publicly available documents (see Discussion and Supplement 1 for details). Finally, TGA asked us to request specific documents, such as clinical study report modules.
      Box 1
      Case 1. FDA's internal disagreement on the authorisation of OROS methylphenidate
      The FDA [

      Center for Drug Evaluation and Research. Drug Approval Package. 21-121/S-017. 27 June 2008. Available from: https://www.accessdata.fda.gov/drugsatfda_docs/nda/2008/021121s017.pdf (accessed 5 Dec 2021).

      ] approved OROS methylphenidate in June 2008 on the basis of the two Janssen sponsored trials, Medori 2005 (5 weeks; 402 participants) [

      NCT00246220. A study of the effectiveness and safety of prolonged-release methylphenidate hydrochloride in adult patients with attention deficit/hyperactivity disorder. Last updated 17 May 2011. Available from: https://clinicaltrials.gov/ct2/show/NCT00246220 (accessed 5 Dec 2021).

      ], and Adler 2006 (7 weeks; 229 participants) [

      NCT00326391. A safety and effectiveness study of methylphenidate hcl extended-release tablets in adults with attention deficit hyperactivity disorder. Last updated 24 May 2011. Available from: https://clinicaltrials.gov/ct2/show/NCT00326391 (accessed 5 Dec 2021).

      ]. FDA's medical reviewer raised concerns regarding the drug's harms, particularly cardiovascular safety, and recommended the conduct of a third trial as a condition for approval (p. 72) [

      Center for Drug Evaluation and Research. Drug Approval Package. 21-121/S-017. 27 June 2008. Available from: https://www.accessdata.fda.gov/drugsatfda_docs/nda/2008/021121s017.pdf (accessed 5 Dec 2021).

      ]. The FDA Team Leader and the Division Director did not agree, among other reasons, because they believed it was not feasible, nor ethical, to conduct more clinical trials. The Division Director, Thomas Laughren stated, “The study proposed by Dr. Mannheim is simply not feasible. It would need to involve hundreds of thousands of patients, would need a placebo arm, and would take years to complete. In the meantime, the labels for Concerta and other drugs in this class already have very strong warning language that alerts prescribers to possible cardiovascular risks. Thus, I do not agree with the need for the study proposed by Dr. Mannheim, and I will not suggest it to the sponsor” (p. 43) [

      Center for Drug Evaluation and Research. Drug Approval Package. 21-121/S-017. 27 June 2008. Available from: https://www.accessdata.fda.gov/drugsatfda_docs/nda/2008/021121s017.pdf (accessed 5 Dec 2021).

      ]. At the time of decision-making, five clinical trials were ongoing or completed that did not appear from the FDA Review [

      Center for Drug Evaluation and Research. Drug Approval Package. 21-121/S-017. 27 June 2008. Available from: https://www.accessdata.fda.gov/drugsatfda_docs/nda/2008/021121s017.pdf (accessed 5 Dec 2021).

      ]. Furthermore, the applicant submitted information on four placebo-controlled trials, but the FDA did not ask for these trials, “The source papers have not been reviewed at the time of this review” (p. 125) [

      Center for Drug Evaluation and Research. Drug Approval Package. 21-121/S-017. 27 June 2008. Available from: https://www.accessdata.fda.gov/drugsatfda_docs/nda/2008/021121s017.pdf (accessed 5 Dec 2021).

      ]. See also Supplement 1, eCase 2.
      Case 2. The missing Casas 2008 trial across OROS methylphenidate authorisations
      Health Canada approved OROS methylphenidate for adult ADHD in April 2008 [

      Health Canada. Concerta Product Monograph. Submission control no. 224296. 17 April 2019. Available from: https://pdf.hres.ca/dpd_pm/00050777.PDF (accessed 5 Dec 2021).

      ] on the basis of one trial, Medori 2005 [

      NCT00246220. A study of the effectiveness and safety of prolonged-release methylphenidate hydrochloride in adult patients with attention deficit/hyperactivity disorder. Last updated 17 May 2011. Available from: https://clinicaltrials.gov/ct2/show/NCT00246220 (accessed 5 Dec 2021).

      ]. In June 2008, FDA approved the drug [

      Center for Drug Evaluation and Research. Drug Approval Package. 21-121/S-017. 27 June 2008. Available from: https://www.accessdata.fda.gov/drugsatfda_docs/nda/2008/021121s017.pdf (accessed 5 Dec 2021).

      ] on the basis of two trials, Medori 2005 [

      NCT00246220. A study of the effectiveness and safety of prolonged-release methylphenidate hydrochloride in adult patients with attention deficit/hyperactivity disorder. Last updated 17 May 2011. Available from: https://clinicaltrials.gov/ct2/show/NCT00246220 (accessed 5 Dec 2021).

      ] and Adler 2006 [

      NCT00326391. A safety and effectiveness study of methylphenidate hcl extended-release tablets in adults with attention deficit hyperactivity disorder. Last updated 24 May 2011. Available from: https://clinicaltrials.gov/ct2/show/NCT00326391 (accessed 5 Dec 2021).

      ]. In January 2009, the TGA [

      Therapeutic Goods Administration. Concerta extended release tables. CCDS181023. Date of revision 21 Oct 2021. Available from: https://www.ebs.tga.gov.au/ebs/picmi/picmirepository.nsf/pdf?OpenAgent&id=CP-2009-PI-00919-3&d=202005111016933 (accessed 5 Dec 2021).

      ] approved the drug based on the same two trials, but they also listed harms’ data from a third study, Casas 2008 (13 weeks; 279 participants) [

      NCT00714688. A study to evaluate effectiveness and safety of prolonged release OROS methylphenidate in adults with attention deficit hyperactivity disorder. Last updated 8 May 2014. Available from: https://clinicaltrials.gov/ct2/show/NCT00714688 (accessed 5 Dec 2021).

      ]. This trial was conducted between February 2008 and April 2009 and (seemingly) ongoing at the time of FDA, Health Canada, and TGA's decisions, despite only appearing in TGA's documents.
      In June 2010, the British regulator, MHRA [

      Medicines and Healthcare Products Regulatory Agency. Public Assessment Report. Concerta XL 27 mg prolonged-release tablets. UK/H/0544/004/MR. Available from: https://mhraproductsproduction.blob.core.windows.net/docs/e16cca8676c72aff9c51ed19f93314a251e61a05 (accessed 5 Dec 2021).

      ], rejected the OROS methylphenidate application on the basis of all three Janssen trials [

      NCT00246220. A study of the effectiveness and safety of prolonged-release methylphenidate hydrochloride in adult patients with attention deficit/hyperactivity disorder. Last updated 17 May 2011. Available from: https://clinicaltrials.gov/ct2/show/NCT00246220 (accessed 5 Dec 2021).

      ,

      NCT00326391. A safety and effectiveness study of methylphenidate hcl extended-release tablets in adults with attention deficit hyperactivity disorder. Last updated 24 May 2011. Available from: https://clinicaltrials.gov/ct2/show/NCT00326391 (accessed 5 Dec 2021).

      ,

      NCT00714688. A study to evaluate effectiveness and safety of prolonged release OROS methylphenidate in adults with attention deficit hyperactivity disorder. Last updated 8 May 2014. Available from: https://clinicaltrials.gov/ct2/show/NCT00714688 (accessed 5 Dec 2021).

      ]. Taking into account dropouts as ‘treatment failures’, the MHRA reviewer commented, “The totality of the data is therefore weak, with one successful, one borderline failure, and one clearly failed trial” (p. 106) [

      Medicines and Healthcare Products Regulatory Agency. Public Assessment Report. Concerta XL 27 mg prolonged-release tablets. UK/H/0544/004/MR. Available from: https://mhraproductsproduction.blob.core.windows.net/docs/e16cca8676c72aff9c51ed19f93314a251e61a05 (accessed 5 Dec 2021).

      ]. The Casas 2008 [

      NCT00714688. A study to evaluate effectiveness and safety of prolonged release OROS methylphenidate in adults with attention deficit hyperactivity disorder. Last updated 8 May 2014. Available from: https://clinicaltrials.gov/ct2/show/NCT00714688 (accessed 5 Dec 2021).

      ] trial was the “clearly failed trial”. It can be speculated whether the FDA and Health Canada had decided differently, if the Casas 2008 trial had appeared in their public documents. See also Supplement 1, eCases 1 to 4.
      Case 3. BfArM's approval of extended-release methylphenidate and the COMPAS trial
      The German drug regulator, BfArM, approved the drug for use in adults in April 2011 [

      Center for Drug Evaluation and Research. Drug Approval Package. Focalin XR. 26 May 2005. Available from: https://www.accessdata.fda.gov/drugsatfda_docs/nda/2005/021802s000TOC.cfm (accessed 5 Dec 2021 ).

      ,

      BfArM Presse. Methylphenidat auch für Erwachsene: BfArM erweitert Zulassung. 15 April 2011. Available from: https://web.archive.org/web/20110901054212/http://www.bfarm.de/DE/BfArM/Presse/mitteil2011/pm02-2011.html?nn=1009636 (accessed 5 Dec 2021).

      ] on the basis of two pivotal trials, Rösler 2004 (24 weeks; 359 participants) [

      NCT00619840. Placebo-Controlled Multi-Centre Double-Blind Trial for Adults With Extended-Release Methylphenidate for ADHD (EMMA). Last updated 21 Feb 2008. Available from: https://clinicaltrials.gov/ct2/show/NCT00619840 (accessed 5 Dec 2021).

      ] and Retz 2008 (8 weeks; 162 participants) [

      NCT00730249. Quality Assurance of Administering Methylphenidate in Adults With Attention Deficit Hyperactivity Disorder (ADHD) - QUMEA (QUMEA). Last updated: 13 January 2010. Available from: https://clinicaltrials.gov/ct2/show/NCT00730249 (accessed 5 Dec 2021).

      ]. These trials were conducted between 2004 to 2006 and 2008 to 2010, respectively, and sponsored by the applicant, Medice. A German publicly funded trial, COMPAS 2007 (52 weeks, 433 participants) [
      • Philipsen A
      • Jans T
      • Graf E
      • Matthies S
      • Borel P
      • Colla M
      • et al.
      Effects of group psychotherapy, individual counseling, methylphenidate, and placebo in the treatment of adult attention-deficit/hyperactivity disorder: A randomized clinical trial.
      JAMA Psychiatry. 2015; 72: 1199-1210
      ], was conducted in the same period (2007 to 2011) and it finished in August 2011 [
      • Philipsen A
      • Jans T
      • Graf E
      • Matthies S
      • Borel P
      • Colla M
      • et al.
      Effects of group psychotherapy, individual counseling, methylphenidate, and placebo in the treatment of adult attention-deficit/hyperactivity disorder: A randomized clinical trial.
      JAMA Psychiatry. 2015; 72: 1199-1210
      ], four months after BfArM's approval. The applicant, Medice, was involved in COMPAS’ trial design and outcome collection [
      • Philipsen A
      • Jans T
      • Graf E
      • Matthies S
      • Borel P
      • Colla M
      • et al.
      Effects of group psychotherapy, individual counseling, methylphenidate, and placebo in the treatment of adult attention-deficit/hyperactivity disorder: A randomized clinical trial.
      JAMA Psychiatry. 2015; 72: 1199-1210
      ], but the trial did not appear from the Public Assessment Report [

      Center for Drug Evaluation and Research. Drug Approval Package. Focalin XR. 26 May 2005. Available from: https://www.accessdata.fda.gov/drugsatfda_docs/nda/2005/021802s000TOC.cfm (accessed 5 Dec 2021 ).

      ]. We asked BfArM whether they were informed about the COMPAS trial prior to approval, to which they responded that the study was ongoing at the time of decision-making (Supplement 1 eTable 6 and Supplement 2). BfArM further informed us that Medice included data from COMPAS in a subsequent ‘Type II variation’ submission, which was an extension of the indication to also include initiation of treatment in adults that have not previously been treated. The type II variation was approved in November 2017 [

      Bundesinstitut für Arzneimittel und Medizinprodukte. Public Assessment Report. Methylphenidate hydrochloride. DE/H/0690/001-007/MR . Updated 19 January 2018. Available from: https://www.geneesmiddeleninformatiebank.nl/Pars/h34030.pdf (accessed 5 Dec 2021).

      ]. See also Supplement 1, eCase 6.

      3.5 Regulatory requirements for new drug applications

      In general, the FDA, TGA, Health Canada, and EMA seemed to require all trials to be submitted but there were important uncertainties in their wording, see Table 4. We enquired with the agencies (Supplement 1 eTable 5 and Supplement 2), and we also sought information on the agency websites (Supplement 1 eTable 6).
      The FDA guideline [

      Center for Drug Evaluation and Research. Guideline for the Format and Content of the Clinical and Statistical Sections of an Application. July 1988. Available from: https://www.fda.gov/media/71436/download (accessed 5 Dec 2021).

      ] and the US legislation [

      Electronic Code of Federal Regulation. Title 21, chapter 1, subchapter d, part 314, subpart B, §314.50. Updated 2 April 2020. Available from: https://www.ecfr.gov/cgi-bin/text-idx?SID=6d2c00e689dbe39f96cc4fe77ce33908&mc=true&node=se21.5.314_150&rgn=div8 (accessed 5 Dec 2021).

      ] specified that all clinical trials “pertinent” to the proposed indication should be included. Health Canada requires applicants to submit a certificate stating, “all pivotal trials necessary to support the proposed indications […] have been provided “ [

      Health Canada. Submission certificate for a NDS, SNDS, SANDS, ANDS, or NC. In effect 1 April 2020. Available from: https://www.canada.ca/content/dam/hc-sc/documents/services/drugs-health-products/drug-products/applications-submissions/forms/certnds_attpdn-2020-eng.doc (accessed 5 Dec 2021).

      ], but their guideline [

      Health Canada. Guidance document: guidance document: preparation of drug regulatory activities in the common technical document (CTD) format. 22 June 2012. Available from: https://www.canada.ca/en/health-canada/services/drugs-health-products/drug-products/applications-submissions/guidance-documents/common-technical-document/preparation-drug-regulatory-activities-common-technical-document-format.html (accessed July 2021).

      ] on the Common Technical Document did not clarify this further. EMA's guideline [

      European Medicines Agency. ICH M4 Common Technical Document for the registration of pharmaceuticals for human use – efficacy. Revision 2. July 2016. Available from: https://www.ema.europa.eu/en/ich-m4e-common-technical-document-registration-pharmaceuticals-human-use-efficacy (accessed 5 Dec 2021).

      ] on the Common Technical Document specified that all “pertinent” data should be submitted, whereas the EU Trial Directive 2001/83 [

      European Commission. Directive 2001/83/EC. 6 November 2001. Available from: https://ec.europa.eu/health//sites/health/files/files/eudralex/vol-1/dir_2001_83_consol_2012/dir_2001_83_cons_2012_en.pdf (accessed 5 Dec 2021).

      ], which formed the legal basis for the content of the Marketing Authorisation Dossier, was more broad and stated, “all clinical trials should be communicated”. Subsequent EU Regulations have seemingly not elaborated on the submission requirements (Supplement 1 eTable 7). The TGA has adopted the European Common Technical Document standards [

      Therapeutic Goods Administration. Mandatory requirements for an effective application. Feb 2018. Available from: https://www.tga.gov.au/publication/mandatory-requirements-effective-application (accessed 5 Dec 2021).

      ,

      Therapeutic Goods Administration. International scientific guidelines adopted in Australia. 5 Aug 2020. Available from: https://www.tga.gov.au/ws-sg-index (accessed 5 Dec 2021).

      ].

      4. Discussion

      4.1 Summary of findings

      To our knowledge, this may be the first report to systematically investigate whether drug regulatory agencies make decisions based on complete or select samples of clinical trials. In our cohort of 13 extended-release methylphenidate applications for ADHD in adults, we identified missing trials in 7 (54%) applications. Previously, it has been reported [
      • Jefferson T
      • Jones MA
      • Doshi P
      • Del Mar CB
      • Hama R
      • Thompson MJ
      • et al.
      Neuroaminidase inhibitors for preventing and treating influenza in adults and children.
      Cochrane Database Syst Rev. 2014; 4CD008965
      ] that the EMA and FDA approved oseltamivir for influenza but “largely ignored” the largest oseltamivir trial. Our findings therefore raise the concern that missing trials in drug applications may constitute a general issue.
      There may be several explanations why some clinical trials did not appear in the public documents: First, the companies may deliberately not include all trials in their applications. Second, the drug regulatory agencies do not unequivocally request that the applicants submit all trials. Third, the agencies do not conduct their own systematic searches of clinical trial registries and databases of published literature but rely on the material submitted by the applicant. Fourth, the agencies apparently do not always check that everything listed in the index has been submitted. Our research group has previously found that appendices were only included for 32 of 70 trials of antidepressants [
      • Sharma T
      • Guski LS
      • Freund N
      • Gøtzsche PC.
      Suicidality and aggression during antidepressant treatment: systematic review and meta-analyses based on clinical study reports.
      BMJ. 2016; 352: i65
      ] and in FDA's medical review of OROS methylphenidate [

      Center for Drug Evaluation and Research. Drug Approval Package. 21-121/S-017. 27 June 2008. Available from: https://www.accessdata.fda.gov/drugsatfda_docs/nda/2008/021121s017.pdf (accessed 5 Dec 2021).

      ], they even noted they had not reviewed all ‘bibliographic references’ submitted by the applicant, see Case 1. Fifth, drawing on a case study of depot aripiprazole's authorisation [
      • Boesen K
      • Naudet F
      • Ioannidis JPA.
      Depot aripiprazole: subverted trials and authorisation of a best-selling drug.
      Available as a preprint at SSRN. 18 March 2021; (accessed 5 Dec 2021): 1-31https://doi.org/10.2139/ssrn.3805790
      ], regulatory agencies may exclude clinical trials from public documents depending on their own assessments. In TGA's Public Assessment Report of depot aripiprazole [

      Therapeutic Goods Administration. Australian Public Assessment Report for Aripiprazole Monohydrate. November 2014. Available from: https://www.tga.gov.au/sites/default/files/auspar-aripiprazole-monohyrate-141125.pdf (accessed 5 Dec 2021).

      ] two pivotal trials were assessed but the agency discarded one of these trials due to outcome switching. In the corresponding Product Information [

      Therapeutic Goods Administration. Product Information. Abilify Maintena. 25 Nov 2014. Available from: https://www.tga.gov.au/sites/default/files/auspar-aripiprazole-monohyrate-141125-pi.pdf (accessed 5 Dec 2021).

      ], the discarded trial did not appear in the ‘Clinical Trials’ section, whereas the discarded trial's data appeared under ‘Adverse Events’. This seems similar to TGA's Product Information on OROS methylphenidate [

      Therapeutic Goods Administration. Concerta extended release tables. CCDS181023. Date of revision 21 Oct 2021. Available from: https://www.ebs.tga.gov.au/ebs/picmi/picmirepository.nsf/pdf?OpenAgent&id=CP-2009-PI-00919-3&d=202005111016933 (accessed 5 Dec 2021).

      ] as described in Case 2. Also here, one of the trials, Casas 2008 [

      NCT00714688. A study to evaluate effectiveness and safety of prolonged release OROS methylphenidate in adults with attention deficit hyperactivity disorder. Last updated 8 May 2014. Available from: https://clinicaltrials.gov/ct2/show/NCT00714688 (accessed 5 Dec 2021).

      ], appeared only in the ‘Adverse Events’ section. One might therefore speculate that TGA discarded the Casas 2008 results, which also seems supported by MHRA's classification of it as “a clearly failed trial” [

      Medicines and Healthcare Products Regulatory Agency. Public Assessment Report. Concerta XL 27 mg prolonged-release tablets. UK/H/0544/004/MR. Available from: https://mhraproductsproduction.blob.core.windows.net/docs/e16cca8676c72aff9c51ed19f93314a251e61a05 (accessed 5 Dec 2021).

      ].
      The regulatory submission requirements seemingly stipulate that all available evidence should be submitted. However, the devil may lie in the detail. FDA and EMA (and subsequently TGA) label the required submitted material as “pertinent” and Health Canada defines them as “all pivotal trials”, Table 4. Whether this means that all data from all available trials must be submitted, or only those trials that the regulatory agency and the applicant have already agreed on during pre-authorisation meetings, seems unclear. This uncertainty may stem from the International Council for Harmonisation's (ICH) guideline [

      ICH Harmonised Guideline. Revision of M4E guideline on enhancing the format and structure of benefit-risk information in ICH efficacy M4E(R2). Step 4 version. 15 June 2016. Available from: https://database.ich.org/sites/default/files/M4E_R2__Guideline.pdf (accessed 5 Dec 2021).

      ] on the Common Technical Document Module 5, which all four agencies rely on. The guideline uses the same terminology and labels the included trials, “pertinent to the efficacy of the medicinal product” [

      ICH Harmonised Guideline. Revision of M4E guideline on enhancing the format and structure of benefit-risk information in ICH efficacy M4E(R2). Step 4 version. 15 June 2016. Available from: https://database.ich.org/sites/default/files/M4E_R2__Guideline.pdf (accessed 5 Dec 2021).

      ].
      We noted that most drug regulatory agencies did not directly address our questions regarding the extent of missing trials. The exception was Health Canada who sent us a clinical study report module 2.7 (eCase 1) and referred us to publicly disclosed clinical study report material (eCase 11), which fully addressed our questions. We understand that regulatory staff does not have authority, or perhaps insight, to explain former decisions made by their superiors. However, our questions regarding the availability of specific trials prior to decision-making seem to constitute basic information that, in our view, should already be in the public domain.

      4.2 Limitations

      We did not register a protocol for this analysis. It was an exploratory project related to our systematic review on extended-release methylphenidate for adult ADHD [
      • Boesen K
      • Danborg PB
      • Gøtzsche PC
      • Jørgensen KJ.
      Extended-release methylphenidate for attention deficit hyperactivity disorder (ADHD) in adults (protocol).
      Cochrane Database Syst Rev. 2017; 11CD012857
      ,

      Boesen K, Paludan-Müller AS, Gøtzsche PC, Jørgensen KJ. Extended-release methylphenidate for attention deficit hyperactivity disorder (ADHD) in adults (review). Cochrane Database Syst Rev (In press).

      ], for which a protocol is available [
      • Boesen K
      • Danborg PB
      • Gøtzsche PC
      • Jørgensen KJ.
      Extended-release methylphenidate for attention deficit hyperactivity disorder (ADHD) in adults (protocol).
      Cochrane Database Syst Rev. 2017; 11CD012857
      ]. This analysis is based on documents obtained through our predefined systematic search, and we therefore do not believe the lack of an additional protocol affected the results.
      It is important to reiterate that our analysis was based on public documents and, with the exception of Health Canada's (and probably PMDA's) approval of OROS methylphenidate and Health Canada's approval of controlled-release methylphenidate, we do not know whether the agencies had access to more data. Critically, the listed evidence in Health Canada's publicly available OROS methylphenidate Product Monograph matched the corresponding clinical study report (eCase 1), whereas the controlled-release methylphenidate Product Monograph did not match the clinical study report material (eCase 11). We would therefore need access to clinical study reports from all drug applications to confirm our results, which could take years to obtain.
      Since our findings were largely based on publicly available documents, this may also be an indicator of the current level of transparency. If our estimates of missing trials are incorrect, it may be due to the regulatory agencies’ lack of disclosure. One could argue it is reasonable to include only those trials supporting the authorised indications in public documents. However, it obscures transparency when excluded trials are not listed, and it can give a distorted impression of the submitted evidence and of the drug's benefits and harms.
      It is important to highlight the variation in the available documents. FDA's Drug Approval Packages and the European Public Assessment Reports are lengthy documents with detailed scientific discussions. Health Canada's Clinical Information Package contains part of the clinical study report, but no regulatory discussions. PMDA's report was largely composed of other agencies’ documents. The Canadian Product Monographs and the Australian Prescriber Information are summaries of product characteristics and not scientific reports.
      Our sample of 13 applications related to one drug is small and it is an important limitation to the generalisability across indications, fields, and drug regulators. We also do not know if we have missed additional rejected applications. We were surprised to find reports related to two rejected applications. EMA publishes information on rejected applications, called ‘refusal assessment reports’ [

      European Medicines Agency. Procedural advice on publication of information on negative opinions and refusals of marketing authorization applications for human medicinal products. 2 May 2013. Available from: https://www.ema.europa.eu/en/documents/regulatory-procedural-guideline/procedural-advice-publication-information-negative-opinions-refusals-marketing-authorisation_en.pdf (accessed 5 Dec 2021).

      ]. To our knowledge, this is not common and other regulators, e.g., the FDA, publishes reports of authorised drugs only [

      US Food and Drug Administration. [email protected]: FDA approved drugs. Available from: https://www.accessdata.fda.gov/scripts/cder/daf/ (accessed 5 Dec 2021).

      ].
      We may have missed some publicly funded trials with industry-involvement. Selective reporting of publicly funded trials is particularly pronounced [
      • Goldacre B
      • DeVito NJ
      • Heneghan C
      • Irving F
      • Bacon S
      • Fleminger J
      • et al.
      Compliance with requirement to report results on the EU Clinical Trials Register: cohort study and web resource.
      BMJ. 2018; 362: k3218
      ,
      • Wieschowski S
      • Riedel N
      • Wollmann K
      • Kahrass H
      • Müller-Ohlraun S
      • Schürmann C
      • et al.
      Result dissemination from clinical trials conducted at German university medical centers was delayed and incomplete.
      J Clin Epidemiol. 2019; 115: 37-45
      ], and sometimes the industry-involvement has not been declared in publications, even when the trial was later used in a registration application [
      • Jureidini JN
      • Doecke CJ
      • Mansfield PR
      • Haby MM
      • Menkes DB
      • Tonkin AL.
      Efficacy and safety of antidepressants for children and adolescents.
      BMJ. 2004; 328: 879-883
      ].

      4.3 Suggestions for improvements

      We identified limitations in the current drug regulatory system with potentially serious consequences for the reliability of drug approvals. Despite our lengthy investigation, we are still not certain on what basis most drugs were approved. It is therefore regrettable that for instance the FDA plans to weaken transparency by phasing out the publication of medical, statistical, and other reviews and substituting them with less informative 'integrated reviews' [
      • Herder M
      • Morten CJ
      • Doshi P.
      Integrated drug reviews at the US Food and drug administration - legal concerns and knowledge lost.
      JAMA Intern Med. 2020; 180: 629-630
      ].
      Drug regulators need to ensure that their decisions are based on all data from all relevant trials and not just a selection of them. The current gaps could be addressed on several levels: It is highly recommendable that regulatory agencies conduct their own systematic searches of trial registries and databases of published literature. According to FDA's manual on how to conduct clinical reviews (section 9.1) [

      Center for Drug Evaluation and Research. Manual of policies and procedures. Good review practice: clinical review template. Effective date 10 Dec 2010. Available from: https://www.fda.gov/media/72472/download (accessed 5 Dec 2021).

      ], reviewers are not obliged to systematically search for evidence; the submission requirements need to be clarified to avoid loopholes and these requirements must be enforced; the pharmaceutical companies need to be held accountable if they fail to submit relevant data or trials, or fail to inform about ongoing or planned trials; finally, it would be sensible to increase collaboration between regulatory agencies. Health Canada and the TGA have already initiated projects on shared drug review assessments with the Singaporean and Swiss authorities [

      Health Canada. Improving the regulatory review of drugs and devices. 26 April 2020. Available from: https://www.canada.ca/en/health-canada/corporate/transparency/regulatory-transparency-and-openness/improving-review-drugs-devices.html (accessed 5 Dec 2021).

      ]. Our recommendations are summarised in Box 2. The European Commission recently solicited input to the revision of the EU general pharmaceutical legislation [

      European Commission. Revision of the EU general pharmaceutical legislation. Not dated. Available from: https://ec.europa.eu/info/law/better-regulation/have-your-say/initiatives/12963-Revision-of-the-EU-general-pharmaceuticals-legislation_en (accessed 5 Dec 2021).

      ]. This may be an opportunity to address some of the issues we highlight in this paper, including clarification of submission requirements and better overviews of assessed clinical trials in drug applications [

      Boesen K, Paludan-Müller AS. Feedback from: meta-research innovation center berlin (METRIC-B). 27 April 2021. Available from: https://ec.europa.eu/info/law/better-regulation/have-your-say/initiatives/12963-Revision-of-the-EU-general-pharmaceuticals-legislation/F2253796_en (accessed 5 Dec 2021).

      ].
      Box 2
      Recommendations to improve the completeness and transparency of regulatory decisions
      1. Regulatory agencies should systematically search clinical trial registries and databases of published literature prior to decision-making to avoid missing clinical trials.
      2. Regulatory agencies should clearly disclose all trials (not just the favourable ones) they assessed prior to decision-making in publicly available documents.
      3. Regulatory submission requirements regarding available clinical trials should be made clearer to avoid loopholes in interpretation.
      4. Regulatory agencies should consider establishing an international website with basic information on drug authorisations (e.g. assessed pivotal and supporting trials, regulatory discussions, concerns raised, and final decision) to enable comparisons, reduce redundancy, and increase transparency.
      5. Pharmaceutical companies should be held accountable if they fail to inform regulatory agencies about all available evidence.
      On a general note, our method for indexing clinical trials based on published reports and clinical trial registries and our identification of publicly available regulatory documents can be applied across drugs and indications. Our research group used a similar approach to map clinical trials of the HPV vaccines [
      • Jørgensen L
      • Gøtzsche PC
      • Jefferson T.
      Index of the human papillomavirus (HPV) vaccine industry clinical study programmes and non-industry funded studies: a necessary basis to address reporting bias in a systematic review.
      Syst Rev. 2018; 7: 8
      ]. Our method also highlights potential solutions to address the current loopholes in drug regulation, Box 2. However, this project should also be considered a ‘work-in-progress’ and it will be updated if new information emerges.

      5. Conclusion

      Based on publicly available regulatory documents, we found that 7 (54%) of 13 regulatory decisions regarding extended-release methylphenidate for adult ADHD were made based on a select sample of clinical trials, although current requirements seem to state that all available trials should be included in drug applications. It will be important to assess larger cohorts of drug approvals to better estimate the prevalence of missing trials in new drug applications. Similarly, drug regulatory agencies may consider employing new protocols to avoid missing clinical trials prior to decision-making.

      Availability of data and material

      All data underlying this project are available from, or referred to, in the supplements. It was an exploratory analysis related to our systematic review and we did not publish a separate protocol for this project.

      Funding

      METRIC-B (KB) is funded by grants from the Einstein Foundation and Stiftung Charité.

      Author contributions

      KB coined the idea, collected the data, drafted the letters to the drug regulators, and wrote the first draft of the paper. KJ and PCG supervised the correspondence with the drug regulators and critically revised the manuscript. All authors approved the final manuscript. KB is the guarantor and attests that all listed authors meet authorship criteria and that no others meeting the criteria have been omitted.

      Acknowledgements

      Asger Sand Paludan-Müller (ASP) extracted data for our systematic review on methylphenidate for adult ADHD, which formed the basis for this analysis.

      Appendix. Supplementary materials

      References

        • Dwan K
        • Altman DG
        • Arnaiz JA
        • Bloom J
        • Chan A-W
        • Cronin E
        • et al.
        Systematic review of the empirical evidence of study publication bias and outcome reporting bias.
        PLOS One. 2008; 3: e3081
        View in Article
        • Dwan K
        • Gamble C
        • Williamson PR
        • Kirkham JJ
        • the Reporting Bias Group
        Systematic review of the empirical evidence of study publication bias and outcome reporting bias — an updated review.
        PLoS ONE. 2013; : e66844
        View in Article
        • Chan AW
        • Hrobjartsson A
        • Haahr MT
        • Gøtzsche PC
        • Altman DG.
        Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles.
        JAMA. 2004; 291: 2457-2465
        View in Article
        • Goldacre B
        • Drysdale H
        • Dale A
        • Milosevic I
        • Slade E
        • Hartley P
        • et al.
        COMPare: a prospective cohort study correcting and monitoring 58 misreported trials in real time.
        Trials. 2019; 20: 118
        View in Article
        • Jefferson T
        • Jones MA
        • Doshi P
        • Del Mar CB
        • Hama R
        • Thompson MJ
        • et al.
        Neuroaminidase inhibitors for preventing and treating influenza in adults and children.
        Cochrane Database Syst Rev. 2014; 4CD008965
        View in Article
        • Jørgensen L
        • Gøtzsche PC
        • Jefferson T.
        Benefits and harms of the human papillomavirus (HPV) vaccines: comparison of trial data from clinical study reports with corresponding trial register entries and journal publications.
        Syst Rev. 2020; 9: 42
        View in Article
        • Munkholm K
        • Paludan-Müller AS
        • Boesen K.
        Considering the methodological limitations in the evidence of antidepressants for depression: a reanalysis of a network meta-analysis.
        BMJ Open. 2019; 9e024886
        View in Article
        • Wieseler B
        • Wolfram N
        • McGauran N
        • Kerekes MF
        • Vervölgyi V
        • Kohlepp P
        • et al.
        Completeness of reporting of patient-relevant clinical trial outcomes: comparison of unpublished clinical study reports with publicly available data.
        PLOS Medicine. 2013; 10e1001536
        View in Article
        • Noury JL
        • Nardo JM
        • Healy D
        • Jureidini J
        • Raven M
        • Tufanaru C
        • et al.
        Restoring Study 329: efficacy and harms of paroxetine and imipramine in treatment of major depression in adolescence.
        BMJ. 2015; 351: h4320
        View in Article
        • Turner EH
        • Matthews AM
        • Linardatos E
        • Tell RA
        • Rosenthal R.
        Selective publication of antidepressant trials and its influence on apparent efficacy.
        N Engl J Med. 2008; 358: 252-260
        View in Article

      11. International Council for Harmonisation of technical requirements for pharmaceuticals for human use. organisation of the common technical document for the registration of pharmaceuticals for human use. M4. Current step 4 version. 15 June 2016. Available from: https://database.ich.org/sites/default/files/M4_R4__Guideline.pdf (accessed 5 Dec 2021).

        View in Article

      12. European Medicines Agency. ICH M4 Common technical document for the registration of pharmaceuticals for human use – efficacy. Revision 2. July 2016. Available from: https://www.ema.europa.eu/en/ich-m4e-common-technical-document-registration-pharmaceuticals-human-use-efficacy (accessed 5 Dec 2021).

        View in Article
        • Doshi P
        • Jefferson T.
        Clinical study reports of randomised controlled trials: an exploratory review of previously confidential industry reports.
        BMJ Open. 2013; 3e002496
        View in Article
        • Gøtzsche PC
        • Jørgensen AW.
        Opening up data at the European medicines agency.
        BMJ. 2011; 342: d2686
        View in Article
        • Sharma T
        • Guski LS
        • Freund N
        • Gøtzsche PC.
        Suicidality and aggression during antidepressant treatment: systematic review and meta-analyses based on clinical study reports.
        BMJ. 2016; 352: i65
        View in Article
        • Paludan-Müller AS
        • Sharma T
        • Rasmussen K
        • Gøtzsche PC.
        Extensive selective reporting of quality of life in clinical study reports and publications of placebo-controlled trials of antidepressants.
        Int J Risk Saf Med. 2021; 32: 87-99
        View in Article
        • Maund E
        • Tendal B
        • Hrobjartsson A
        • Jørgensen KJ
        • Lundh A
        • Schroll J
        • et al.
        Benefits and harms in clinical trials of duloxetine for treatment of major depressive disorder: comparison of clinical study reports, trial registries, and publications.
        BMJ. 2014; 348: g3510
        View in Article

      18. Restoring invisible and abandoned trials. Regulatory resources. Not dated. Available from: https://restoringtrials.org/regulatory-resources/ (accessed 5 Dec 2021).

        View in Article
        • Boesen K
        • Danborg PB
        • Gøtzsche PC
        • Jørgensen KJ.
        Extended-release methylphenidate for attention deficit hyperactivity disorder (ADHD) in adults (protocol).
        Cochrane Database Syst Rev. 2017; 11CD012857
        View in Article

      20. Boesen K, Paludan-Müller AS, Gøtzsche PC, Jørgensen KJ. Extended-release methylphenidate for attention deficit hyperactivity disorder (ADHD) in adults (review). Cochrane Database Syst Rev (In press).

        View in Article
        • von Elm E
        • Altman DG
        • Egger M
        • Pocock SJ
        • Gøtzsche PC
        • Vandenbroucke JP
        • et al.
        The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies.
        PLoS Med. 2007; 4: e296
        View in Article
        • Regulation No. 536/2014 of the European Parliament and of the Council
        Official J Euro Union. 2014; (27 May 2014. Available from:) (accessed 5 Dec 2021): L158/1
        https://ec.europa.eu/health/sites/default/files/files/eudralex/vol-1/reg_2014_536/reg_2014_536_en.pdf
        View in Article

      23. European Medicines Agency. Clinical Trial Regulation. Not dated. Available from: https://www.ema.europa.eu/en/human-regulatory/research-development/clinical-trials/clinical-trial-regulation (accessed 5 Dec 2021).

        View in Article

      24. Health Canada. Concerta Product Monograph. Submission control no. 224296. 17 April 2019. Available from: https://pdf.hres.ca/dpd_pm/00050777.PDF (accessed 5 Dec 2021).

        View in Article

      25. Center for Drug Evaluation and Research. Drug Approval Package. 21-121/S-017. 27 June 2008. Available from: https://www.accessdata.fda.gov/drugsatfda_docs/nda/2008/021121s017.pdf (accessed 5 Dec 2021).

        View in Article

      26. Therapeutic Goods Administration. Concerta extended release tables. CCDS181023. Date of revision 21 Oct 2021. Available from: https://www.ebs.tga.gov.au/ebs/picmi/picmirepository.nsf/pdf?OpenAgent&id=CP-2009-PI-00919-3&d=202005111016933 (accessed 5 Dec 2021).

        View in Article

      27. Medicines and Healthcare Products Regulatory Agency. Public Assessment Report. Concerta XL 27 mg prolonged-release tablets. UK/H/0544/004/MR. Available from: https://mhraproductsproduction.blob.core.windows.net/docs/e16cca8676c72aff9c51ed19f93314a251e61a05 (accessed 5 Dec 2021).

        View in Article

      28. Pharmaceuticals and Medical Devices Agency. Summary of application materials. 17 January 2014. Available from: https://www.pmda.go.jp/drugs/2013/P201300158/index.html (accessed 5 Dec 2021).

        View in Article

      29. Bundesinstitut für Arzneimittel und Medizinprodukte. Public Assessment Report Summary [Öffentlicher Bewertungsbericht]. Medikinet Adult. 18 July 2011. Available from: https://portal.dimdi.de/amispb/doc/2011/07/25/2163890/OBFMBD5B836E01CC4AAA.rtf (accessed 5 Dec 2021).

        View in Article

      30. Center for Drug Evaluation and Research. Drug Approval Package. Focalin XR. 26 May 2005. Available from: https://www.accessdata.fda.gov/drugsatfda_docs/nda/2005/021802s000TOC.cfm (accessed 5 Dec 2021 ).

        View in Article
        • Medicines Evaluation Board. Public Assessment Report Ritalin LA
        NL License RVG. 2021; 116377 (20 October 2016. Available from:) (accessed 5 Dec 2021): 116379-116382
        https://www.geneesmiddeleninformatiebank.nl/Pars/h116382.pdf
        View in Article
        • Medicines and Healthcare products Regulatory Agency
        Public Assessment Report. Ritalin XL. UK License. 2018; (. No: 00101/0973-0977. March 2018. Available from: https://mhraproductsproduction.blob.core.windows.net/docs/4781baf366b1fafd0ea203962ccb54faadcdcfcc (accessed 5 Dec 2021))
        View in Article

      33. Therapeutic Goods Administration. Australian Product Information. Ritalin 10/Ritalin LA. Latest amendment 14 Aug 2020. Available from: https://www.ebs.tga.gov.au/ebs/picmi/picmirepository.nsf/pdf?OpenAgent&id=CP-2010-PI-03175-3 (accessed 5 Dec 2021).

        View in Article

      34. Health Canada. Product Monograph. Foquest. Submission Control No: 214860. 1 March 2019. Available from: https://pdf.hres.ca/dpd_pm/00049955.PDF (accessed 5 Dec 2021).

        View in Article

      35. Health Canada. Clinical Information. Foquest – submission control no. 187330. Release date 29 May 2020. Available from: https://clinical-information.canada.ca/ci-rc/item/187330. [Accessed 5 Dec 2021].

        View in Article

      36. Center for Drug Evaluation and Research. Drug Approval Package. Adhansia XR. 27 Feb 2019. Available from: https://www.accessdata.fda.gov/drugsatfda_docs/nda/2019/212038Orig1s000TOC.cfm (accessed 5 Dec 2021).

        View in Article

      37. Health Canada. Product Monograph. Biphentin. Revision 14 January 2020. Available from: https://pdf.hres.ca/dpd_pm/00054671.PDF (accessed 5 Dec 2021).

        View in Article

      38. NCT00246220. A study of the effectiveness and safety of prolonged-release methylphenidate hydrochloride in adult patients with attention deficit/hyperactivity disorder. Last updated 17 May 2011. Available from: https://clinicaltrials.gov/ct2/show/NCT00246220 (accessed 5 Dec 2021).

        View in Article

      39. NCT00326391. A safety and effectiveness study of methylphenidate hcl extended-release tablets in adults with attention deficit hyperactivity disorder. Last updated 24 May 2011. Available from: https://clinicaltrials.gov/ct2/show/NCT00326391 (accessed 5 Dec 2021).

        View in Article

      40. NCT00714688. A study to evaluate effectiveness and safety of prolonged release OROS methylphenidate in adults with attention deficit hyperactivity disorder. Last updated 8 May 2014. Available from: https://clinicaltrials.gov/ct2/show/NCT00714688 (accessed 5 Dec 2021).

        View in Article

      41. BfArM Presse. Methylphenidat auch für Erwachsene: BfArM erweitert Zulassung. 15 April 2011. Available from: https://web.archive.org/web/20110901054212/http://www.bfarm.de/DE/BfArM/Presse/mitteil2011/pm02-2011.html?nn=1009636 (accessed 5 Dec 2021).

        View in Article

      42. NCT00619840. Placebo-Controlled Multi-Centre Double-Blind Trial for Adults With Extended-Release Methylphenidate for ADHD (EMMA). Last updated 21 Feb 2008. Available from: https://clinicaltrials.gov/ct2/show/NCT00619840 (accessed 5 Dec 2021).

        View in Article

      43. NCT00730249. Quality Assurance of Administering Methylphenidate in Adults With Attention Deficit Hyperactivity Disorder (ADHD) - QUMEA (QUMEA). Last updated: 13 January 2010. Available from: https://clinicaltrials.gov/ct2/show/NCT00730249 (accessed 5 Dec 2021).

        View in Article
        • Philipsen A
        • Jans T
        • Graf E
        • Matthies S
        • Borel P
        • Colla M
        • et al.
        Effects of group psychotherapy, individual counseling, methylphenidate, and placebo in the treatment of adult attention-deficit/hyperactivity disorder: A randomized clinical trial.
        JAMA Psychiatry. 2015; 72: 1199-1210
        View in Article

      45. Bundesinstitut für Arzneimittel und Medizinprodukte. Public Assessment Report. Methylphenidate hydrochloride. DE/H/0690/001-007/MR . Updated 19 January 2018. Available from: https://www.geneesmiddeleninformatiebank.nl/Pars/h34030.pdf (accessed 5 Dec 2021).

        View in Article

      46. Center for Drug Evaluation and Research. Guideline for the Format and Content of the Clinical and Statistical Sections of an Application. July 1988. Available from: https://www.fda.gov/media/71436/download (accessed 5 Dec 2021).

        View in Article

      47. Electronic Code of Federal Regulation. Title 21, chapter 1, subchapter d, part 314, subpart B, §314.50. Updated 2 April 2020. Available from: https://www.ecfr.gov/cgi-bin/text-idx?SID=6d2c00e689dbe39f96cc4fe77ce33908&mc=true&node=se21.5.314_150&rgn=div8 (accessed 5 Dec 2021).

        View in Article

      48. Health Canada. Submission certificate for a NDS, SNDS, SANDS, ANDS, or NC. In effect 1 April 2020. Available from: https://www.canada.ca/content/dam/hc-sc/documents/services/drugs-health-products/drug-products/applications-submissions/forms/certnds_attpdn-2020-eng.doc (accessed 5 Dec 2021).

        View in Article

      49. Health Canada. Guidance document: guidance document: preparation of drug regulatory activities in the common technical document (CTD) format. 22 June 2012. Available from: https://www.canada.ca/en/health-canada/services/drugs-health-products/drug-products/applications-submissions/guidance-documents/common-technical-document/preparation-drug-regulatory-activities-common-technical-document-format.html (accessed July 2021).

        View in Article

      50. European Medicines Agency. ICH M4 Common Technical Document for the registration of pharmaceuticals for human use – efficacy. Revision 2. July 2016. Available from: https://www.ema.europa.eu/en/ich-m4e-common-technical-document-registration-pharmaceuticals-human-use-efficacy (accessed 5 Dec 2021).

        View in Article

      51. European Commission. Directive 2001/83/EC. 6 November 2001. Available from: https://ec.europa.eu/health//sites/health/files/files/eudralex/vol-1/dir_2001_83_consol_2012/dir_2001_83_cons_2012_en.pdf (accessed 5 Dec 2021).

        View in Article

      52. Therapeutic Goods Administration. Mandatory requirements for an effective application. Feb 2018. Available from: https://www.tga.gov.au/publication/mandatory-requirements-effective-application (accessed 5 Dec 2021).

        View in Article

      53. Therapeutic Goods Administration. International scientific guidelines adopted in Australia. 5 Aug 2020. Available from: https://www.tga.gov.au/ws-sg-index (accessed 5 Dec 2021).

        View in Article
        • Boesen K
        • Naudet F
        • Ioannidis JPA.
        Depot aripiprazole: subverted trials and authorisation of a best-selling drug.
        Available as a preprint at SSRN. 18 March 2021; (accessed 5 Dec 2021): 1-31https://doi.org/10.2139/ssrn.3805790
        View in Article

      55. Therapeutic Goods Administration. Australian Public Assessment Report for Aripiprazole Monohydrate. November 2014. Available from: https://www.tga.gov.au/sites/default/files/auspar-aripiprazole-monohyrate-141125.pdf (accessed 5 Dec 2021).

        View in Article

      56. Therapeutic Goods Administration. Product Information. Abilify Maintena. 25 Nov 2014. Available from: https://www.tga.gov.au/sites/default/files/auspar-aripiprazole-monohyrate-141125-pi.pdf (accessed 5 Dec 2021).

        View in Article

      57. ICH Harmonised Guideline. Revision of M4E guideline on enhancing the format and structure of benefit-risk information in ICH efficacy M4E(R2). Step 4 version. 15 June 2016. Available from: https://database.ich.org/sites/default/files/M4E_R2__Guideline.pdf (accessed 5 Dec 2021).

        View in Article

      58. European Medicines Agency. Procedural advice on publication of information on negative opinions and refusals of marketing authorization applications for human medicinal products. 2 May 2013. Available from: https://www.ema.europa.eu/en/documents/regulatory-procedural-guideline/procedural-advice-publication-information-negative-opinions-refusals-marketing-authorisation_en.pdf (accessed 5 Dec 2021).

        View in Article

      59. US Food and Drug Administration. [email protected]: FDA approved drugs. Available from: https://www.accessdata.fda.gov/scripts/cder/daf/ (accessed 5 Dec 2021).

        View in Article
        • Goldacre B
        • DeVito NJ
        • Heneghan C
        • Irving F
        • Bacon S
        • Fleminger J
        • et al.
        Compliance with requirement to report results on the EU Clinical Trials Register: cohort study and web resource.
        BMJ. 2018; 362: k3218
        View in Article
        • Wieschowski S
        • Riedel N
        • Wollmann K
        • Kahrass H
        • Müller-Ohlraun S
        • Schürmann C
        • et al.
        Result dissemination from clinical trials conducted at German university medical centers was delayed and incomplete.
        J Clin Epidemiol. 2019; 115: 37-45
        View in Article
        • Jureidini JN
        • Doecke CJ
        • Mansfield PR
        • Haby MM
        • Menkes DB
        • Tonkin AL.
        Efficacy and safety of antidepressants for children and adolescents.
        BMJ. 2004; 328: 879-883
        View in Article
        • Herder M
        • Morten CJ
        • Doshi P.
        Integrated drug reviews at the US Food and drug administration - legal concerns and knowledge lost.
        JAMA Intern Med. 2020; 180: 629-630
        View in Article

      64. Center for Drug Evaluation and Research. Manual of policies and procedures. Good review practice: clinical review template. Effective date 10 Dec 2010. Available from: https://www.fda.gov/media/72472/download (accessed 5 Dec 2021).

        View in Article

      65. Health Canada. Improving the regulatory review of drugs and devices. 26 April 2020. Available from: https://www.canada.ca/en/health-canada/corporate/transparency/regulatory-transparency-and-openness/improving-review-drugs-devices.html (accessed 5 Dec 2021).

        View in Article

      66. European Commission. Revision of the EU general pharmaceutical legislation. Not dated. Available from: https://ec.europa.eu/info/law/better-regulation/have-your-say/initiatives/12963-Revision-of-the-EU-general-pharmaceuticals-legislation_en (accessed 5 Dec 2021).

        View in Article

      67. Boesen K, Paludan-Müller AS. Feedback from: meta-research innovation center berlin (METRIC-B). 27 April 2021. Available from: https://ec.europa.eu/info/law/better-regulation/have-your-say/initiatives/12963-Revision-of-the-EU-general-pharmaceuticals-legislation/F2253796_en (accessed 5 Dec 2021).

        View in Article
        • Jørgensen L
        • Gøtzsche PC
        • Jefferson T.
        Index of the human papillomavirus (HPV) vaccine industry clinical study programmes and non-industry funded studies: a necessary basis to address reporting bias in a systematic review.
        Syst Rev. 2018; 7: 8
        View in Article

      Article info

      Publication history

      Published online: November 06, 2021
      Accepted: October 30, 2021

      Footnotes

      Conflict of interest: None.

      Identification

      DOI: https://doi.org/10.1016/j.jclinepi.2021.10.027

      Copyright

      © 2021 The Authors. Published by Elsevier Inc.

      User license

      Creative Commons Attribution (CC BY 4.0) |
      How you can reuse Information Icon
      Creative Commons Attribution (CC BY 4.0)

      Permitted

      • Read, print & download
      • Redistribute or republish the final article
      • Text & data mine
      • Translate the article
      • Reuse portions or extracts from the article in other works
      • Sell or re-use for commercial purposes

      Elsevier's open access license policy

      ScienceDirect

      Access this article on ScienceDirect

      Linked Article

      The content on this site is intended for healthcare professionals.


      We use cookies to help provide and enhance our service and tailor content. To update your cookie settings, please visit the Cookie Preference Center for this site.
      Copyright © 2023 Elsevier Inc. except certain content provided by third parties.


      RELX