Missing trials in drug regulatory dossiers may have good reasons, but should be predefined and transparent

Boesen and colleagues have done a tremendous job in addressing the question: when I have a list of clinical trials of a class of long-acting medicinal products containing methylphenidate, and I want to see whether these trials also pop-up in the regulatory dossiers that are assumed to deal with the licensing process of these products for use in adult patients with ADHD, what do I see? []. At first glance a very simple and logical question for everybody interested in how regulatory processes function and whether these processes contribute to public health and patient safety. Boesen and colleagues show however that this is not by any means a simple question to answer. They report about an odyssean, almost forensic, journey across clinical trial registers, scientific literature, regulatory dossiers, freedom of information procedures, public assessment reports, extensive correspondence with regulatory authorizes, and not to forget translated texts of regulatory websites. The study is about an analysis of 18 trials, 13 drug applications, 4 different drug formulations, (most likely) 4 sponsors and 7 regulatory authorities. The authors had a legitimate reason to study this topic and their methodological approach, i.e. comparing two distributions and to look for the overlap does not seem too complex. But it turned out differently. A few reflections.

Firstly the case selection for the study of long-acting methylphenidate indicated for adult patients with ADHD. From the very beginning, about two decades ago, this topic has been heavily debated and contested by various stakeholders [,. Briefly summarized, the clinical evidence that methylphenidate may work in adult patients with ADHD has never been very convincing, for various reasons. To mention a few, unclear underlying pharmacology and pharmacokinetics, heterogeneity of the patient population to be studied, rather poor diagnostics and practical difficulties to conducting robust clinical trials. More importantly there has been a whole array of different views in the medical community of psychiatrists and patient organizations about the therapeutic value that methylphenidate indicated for adult patients with ADHD could bring. Apart from the question whether methylphenidate in adults has any efficacy benefit, the safety chapter of such a therapy has always been rocky and challenging. Concerns about cardiovascular, cerebrovascular and psychiatric adverse effects in the adult populations have been raised many times, and are well documented [,. These concerns have not disappeared despite a large number of pharmacovigilance and postmarketing studies. Some studies required by regulatory authorities are still ongoing and results are awaited. Boesen and colleagues report about these.

The study makes the impact of this erratic context very clear. Complex discussions between and within regulatory agencies, differences in decisions about the indication, i.e. adult patients with ADHD or adult patients with ADHD with symptoms present before age 7 years. The latter refers to continuing use of methylphenidate by patients who started treatment in early childhood but who have reached the age of 18. And not to forget, in Europe long-acting methylphenidate products have been regulated by National Competent Authorities (e.g. MHRA, Bfarm, MEB) with only a limited role of the EMA in the primary licensing process.

Essentially, most regulatory systems in the world work in an applicant-based fashion when it comes to granting a license to bring a medicinal product to the clinic. The applicant, i.e. the company, builds a dossier on quality, safety and efficacy data and is legally obliged to justify a positive benefit-risk of the product for a given purpose (indication) in a well-defined population. All the data, analyses and narratives in the dossier are there to support that justification. It's the task of the applicant to make the case, to convince, to provide sufficient data, logic and reasoning why a licence for a certain medicinal product is justified, both scientifically and legally. The regulators have to review all the provided evidence critically, to dig into all the data, tables and graphs, to challenge the claims of the applicant and to come after several rounds of for- and backwards discussions, requests for more analyses or studies, to a positive or negative decision on granting a license. And all these steps are regulated in terms of evidence requirements, time lines and how to structure the dossier (i.e. Common Technical Document). If the authorities are not convinced about a positive benefit-risk, they have to reject the application. And when the applicant is not happy with the rejection they may go to court.

It is common practice that applicants virtually never get everything they want. There is always downsizing on claims by the regulators. And for good reasons. For instance, companies often oversell the evidence to justify a broad indication. We have shown for oncology products for instance that the regulators respond to this with restrictions in the label, i.e. more narrow indication or request for extra data and evidence to justify the claim []. This feature of restricting the indication is also seen in the case of long-acting methylphenidate indicated for adult patients with ADHD. The labels for OROS methylphenidate products authorized by Health Canada, TGA, FDA and MRHA imply for the indication ADHD in adults the presence of hyperactive-impulsive or inattentive symptoms that were present before age 7 years. Interestingly, PMDA in Japan decided on January 17, 2014, for an unrestricted adult indication: ‘A drug with a new indication and a new dosage in an additional dosage form indicated for the treatment of attention deficit/hyperactivity disorder (AD/HD) in adults’.

But what about the risk of selective inclusion or exclusion of clinical trials in the dossier? Boesen and colleagues express their concerns here, for obvious reasons. Selective reporting and use of clinical data is well documented [,. The authors conclude that in 7/13 applications for long-acting medicinal products containing methylphenidate clinical trials they tagged from a systematic review, have not been reported or could be flagged otherwise as missing.

Is 7/13 an over- or underestimate? I am afraid the latter. Step 2 in the Methods section says ‘databases of publicly available documents’. But there have been many more applications for long-acting methylphenidate products processed by the 7 authorities. For instance Germany and the Netherlands also have regulated OROS methylphenidate products and these are not in the analysis of the paper. When the 13 applications have been selected based on public accessibility, there is good reason to have concerns about an underestimate.

Intriguingly, the problem with the missing trials flagged in this study is mainly seen in relation to two products, i.e. OROS methylphenidate and one extended release methylphenidate, with (most likely) two applicants. In addition the missing trials are not presented consistently. For example, the five applications for OROS methylphenidate products show a mixed picture of missing trials.

Another interesting finding is that only 3/18 trials (Winhusen, 2005, Goodman, 2009, Weisler, 2009) were not submitted to any of the studied regulatory authorities. That 15/18 trials are seen by at least one authority may be good news. It is common practice, for sure in Europe, that the clinical assessors and experts in a certain field from various agencies meet on a regular basis and discuss the available and relevant science in a certain clinical area. Actually, that's one of the aims and ambitions of regulatory science, i.e. keeping up the agencies with the most recent science [,. The chance that a relevant trial or other data critical for the field would have been missed is most likely very small. Data sharing and oversight, cross-dossier learning, reliance and other types of regulatory collaboration are also a top priority of groups like the International Coalition of Medicines Regulatory Authorities (ICMRA) and in Europe, the Coordination Group for Mutual Recognition and Decentralized Procedures - Human (CMDh) [,.

A key question is why a trial is missing. The authors also speculate on that question. One perspective could help for a better understanding. A clinical trial is only relevant for an application when patients to be treated according to the claimed indication are reflected in the trial population. To illustrate, all applications OROS methylphenidate products included the Medori trial (2005) and at least 3 the Casas trial (2008). These trials had as an inclusion criterion that adults with ADHD should have been symptomatic before the age 7 years. The labels for methylphenidate approved by Health Canada, TGA, FDA and MHRA also reflect this restricted use, interestingly not the PMDA label.

In contrast, the Biederman (2003) and Winhusen (2005) trials for instance had no such population restrictions. They included also adults with ADHD with no history of disease, thereby making these trials of little relevance for considering them for the restricted indication because these studies were conducted in the wrong population. The Winhusen trial (2005) was also more directed at prolonged abstinence for smokers in adult patients with ADHD than the treatment effect on ADHD itself. One may argue that these trials should have been included and considered anyhow for safety reasons, for example. However the limited trial size and the wrong study population mean that they would have contributed very little to any scientific weighing or judgement.

Boesen and colleagues make a good point on the rather limited role of academic, non-industry sponsored, trials in building regulatory dossiers. And there are many reasons for this missed opportunity, not least because many academic trials are not designed and conducted in a fashion that would make them fit for regulatory purpose. The new Clinical Trial Directive in Europe, but also elsewhere, is hopefully a step forward []. It's too early to take stock, but when the ambitions of the directive become reality, the differences between industry and academic clinical trials will most likely become smaller. This will be great progress.

All these reflections on the characteristics of the missing trials may be considered irrelevant given that Boesen and colleagues advocate that all trials should be included and disclosed, and that regulators should actively search registries and databases to ensure applicants comply with this requirement. Indeed, companies are already obliged to submit all clinical trials that are pivotal, pertinent or necessary to support the claim that the benefit-risk of the use of the product in the proposed indication is positive. But in the regulations there is very little on what pivotal, pertinent or necessary really means. It's very much high level wording and case by case work. The authors conclude ‘…. the wording is ambiguous’, and they are right. However, arguing for actively searching for ‘all clinical trials’ is rather meaningless, and would give false hopes, when it does not go hand in hand with transparent methodology and justification on selection of trials, both for the companies and authorities, and not to forget academic groups. Excluding irrelevant trials from a regulatory dossier is in my view a must, for scientific and efficiency reasons. But it should be done in a predefined and transparent fashion. The study of Boesen and colleagues shows examples of non-transparent and unwarranted exclusion of trials. Here regulatory science can learn a lot from systematic reviews where robust and transparent search, inclusion and exclusion methods have been developed over the last decades, with concrete guidelines on how to apply these. After all, appropriate searching and study selection are also pivotal for the production of high quality systematic reviews.

Transparency on how regulatory processes work and how decisions are made is unfinished business, albeit there have been numerous initiatives on policy change, implemented and underway [,. I expect that the future on trial transparency will be a better one, avoiding the need for such heroic and troublesome data digging as Boesen and colleagues have conducted. For the moment the study remains important in adding to the awareness and need for continuous learning and improvement. Regulatory science is too important to leave it to the regulators [].