Highlights
- •Challenges of rapid COVID-19 drug development risk to compromise good scientific practice
- •Access to patient-level data especially urgent to address risks of protocol violations and rapid reviews.
- •Open access to data enables transparency, secondary analyses, and meta-analyses.
- •Politicization of scientific reports will be reduced by cross-checking and multilateral analyses.
1. Introduction
In the current pandemic of COVID-19, intense research efforts are underway to identify effective interventions as soon as possible. This race against time poses an enormous challenge. To maximize the chances of success, data from clinical trials should be used in the most efficient, open, and flexible and time-sensitive manner.
The pressing need to find treatments and vaccines against the virus has led to accelerated regulatory approval processes, such as the emergency program by the Food and Drug Administration (FDA) that provides ultrarapid protocol reviews for clinical trials conducted within 24 hours of submission. Yet, such a speed comes at a cost, and conducting clinical trials in the COVID-19 crisis means to weigh limited resources against the demand to swiftly conclude the completion of the trials. The FDA recognized that shortage of clinical staff, travel restrictions, and interruptions of supply chains will inevitably lead to protocol violations in clinical trials on the COVID-19 treatment [
[1]
]. These difficulties also may not allow recruiting promptly the anticipated number of trial participants. The time pressure and difficult circumstances to conduct clinical trials [[2]
] increase the chances of spurious results, false decisions, and reversals of decisions. For example, the FDA granted a fast-track emergency use authorization for hydroxychloroquine and chloroquine on March 28, 2020 and revoked this authorization on June 15, 2020, that is, within 1 and a half months [[3]
].In these difficult and rapidly changing circumstances, good scientific practice, reproducibility, and transparency are essential principles that must guide clinical trials to adequately inform medical decision-making and keep public trust.
2. Open access to enhance reproducibility and trust in clinical trial results
Open access to patient-level data from clinical trial data is a key instrument to enable reanalysis and validation of data analyses, and thus foster transparency, reproducibility, and trust in the clinical research findings. Moreover, beyond allowing for replication of data analyses by peer researchers, open access to clinical trial data allows to conduct secondary analyses, which are useful in understanding important clinical outcomes and predictors thereof [
[4]
]. For some treatments, it is possible that any benefits, if present, may pertain to subsets of patients and settings differing from those originally defined the eligibility criteria of the clinical trials. To explore treatment effects contingent on patient characteristics or settings, data would need to be efficiently retrieved and combined from multiple trials. Such reanalyses would be difficult to promptly complete, unless the key patient-level data from each trial are readily available for reuse in cross-trial analyses incorporating rolling meta-analyses. For example, as of the writing of this commentary (October 2), the only intervention that has been shown to prolong survival in COVID-19 in a randomized trial is dexamethasone. However, the trialists of the RECOVERY trial who detected this benefit already focused on the fact that dexamethasone seems to decrease mortality risk by a third in people who are intubated and receive invasive mechanical ventilation and by a fifth among those who require oxygen without invasive mechanical ventilation, but has no benefit (and even a trend for increased mortality) in patients who do not require oxygen [[5]
]. Given the spurious track record of subgroup analyses in clinical trials in general [[6]
], such claims would need to be verified ideally across several trials and rigorous meta-analyses thereof. A rapid meta-analysis of seven trials has confirmed the benefit of dexamethasone in critically ill patients [[7]
], and similar meta-analyses would be worthwhile performing for patient with less serious illness.The COVID-19 landscape of clinical trials is already very wide, but also fragmented, currently exceeding a thousand registered trials (for an emerging list, see https://covid-evidence.org/database) that evolve in an uncoordinated way [
[8]
]. It is unclear how many of these trials are on track for enrolling the number of patients that they had anticipated to enroll. Many trials may fall short of enrolling their target sample size because of the time pressure and because the lockdown has drained access to patients. For trials aiming to recruit patients with severe disease treated at an intensive care unit, the available numbers may be very small in many locations. If several trials are abandoned currently or in the near future because of futility, it would be a pity to waste whatever data they have accumulated. These data may still be useful to consider along with those accrued from other similar or complementary trials and may offer guidance for treatment in countries where the epidemic waves are still very active or for potential second waves and resurgences of the epidemic in various countries. Open access to clinical trial data that allows for a rigorous patient-level accumulation of the composite evidence may offer advantages for transparent and accelerated drug development in the current COVID-19 crisis and will be conducive to open synthesis of research findings in reviews and meta-analyses [[9]
].3. Benefits of open access outweigh risks in current COVID-19 crises
The discussion of making patient-level clinical data trials openly accessible has been going on for over a decade, but without much tangible progress until recently. The concept gained more traction with the publication data sharing statements recommended by the International Committee of Medical Journal Editors [
[10]
,[11]
]. An important advancement is that clinical trials are now required to provide a data sharing plan already by the time of the registration, which is a pressing issue given that only a fraction of clinical trial units have any data sharing plan in place [[12]
]. Yet, before the advent of the COVID-19 crisis, there was no sense of urgency to share, and the requirements have fallen short of making the sharing of patient-level clinical trial data obligatory. In fact, out of 487 trials published by JAMA, Lancet, and New England Journal of Medicine since July 1, 2018, only two had provided deidentified patient-level data as of April 10, 2020 [[13]
]. A major concern against sharing deidentified participant-level data from clinical trials is that the investigators who collected the data are not sufficiently credited. However, a hiatus of rendering all patient-level data fully available could reasonably be extended until the acceptance for publication of the first preregistered analysis by the original authors. Moreover, for many trials, for example, those that stop because of futility, the authors themselves may wish to make their data available immediately. This would allow their data to be used with proper credit to them, instead of having them wasted.Another major concern is centered on data protection issues including reidentification of data. However, several anonymization procedures have been proposed, and metrics for the quantitative assessment of the risk of reidentification have been developed [
[14]
]. Although entirely irreversible anonymization of data remains difficult, a balance between the benefit of sharing the data and risk of personal harm by disclosure must be achieved. We argue that the unprecedented urgency for the identification of effective drugs against COVID-19 has shifted the balance toward potential benefit arising from the wider availability of the patient-level data. From the perspective of patients, an overwhelming majority of participants recruited into clinical trials have also voiced support for making their data accessible to the research community [European Medicines Agency
Data anonymisation - a key enabler for clinical data sharing.
Data anonymisation - a key enabler for clinical data sharing.
2017
https://www.ema.europa.eu/en/documents/report/report-data-anonymisation-key-enabler-clinical-data-sharing_en.pdf
Date accessed: April 16, 2020
[15]
].4. Open access is an urgent need as shown by recent article retractions
The need for full transparency and leveraging of the expertise of peer researchers becomes evident against the background of the highly controversial example of emergency use authorization of the drugs hydroxychloroquine and chloroquine in the COVID-19 crisis in the United States and other countries. The available evidence for the effectiveness of the drug for the treatment of COVID-19 that fueled these authorizations was debatable. It was based on data from studies that were small, missed placebo controls, lacked blinding, and/or remained inconclusive due to compromising confounding variables [
[16]
]. Open access to data from already existing trials might have significantly enhanced transparency and gained insight into the reproducibility of the reported results.The hydroxychloroquine story offers a prime example on how this entire clinical research agenda is operating precariously in a shifting sand environment and how time-sensitive COVID-19–related evidence can be. The publication [
[17]
] and retraction [[18]
] of a large multinational observational study reporting increased mortality risk in patients with COVID-19 treated with hydroxychloroquine or chloroquine demonstrated the fragility of the COVID-19 clinical research ecosystem and the need for openness and transparency. Many researchers quickly pointed out inconsistencies in the data, lack of control for potential confounding variables, and insufficient transparencies concerning the data and analysis methods used in the contentious study []. Those comments by the scientific community triggered the publication of an expression of concern by the editors of The Lancet, calling for a cautious interpretation of the study's results until the provenance and validity of the data are clarified [[20]
]. A few days later, three authors retracted the article because they were unable to verify the data that had supposedly been collected and used for this analysis by a company called Surgisphere. Similarly, a New England Journal of Medicine article, where the same company was involved, was also retracted [[21]
]. In response to the retraction of the recent study on hydroxychloroquine, the editors of the The Lancet have recently changed the peer review policy, requiring now that at least one other academic co-author of the article vouches for the veracity of the data [- Mehra M.R.
- Desai S.S.
- Kuy S.
- Henry T.D.
- Patel A.N.
Retraction: Cardiovascular disease, drug Therapy, and mortality in Covid-19.
N Engl J Med. 2020; ([published online])https://doi.org/10.1056/NEJMoa2007621
[22]
]. This is certainly an encouraging step forward. Yet, the urgency of open access to patient-level data remains given that the necessary level of transparency and full leverage of information from clinical trials can only be achieved if the original data are shared more widely within the research community.After the retraction of that observational study on (hydroxy-)chloroquine, a large trial on COVID-19, RECOVERY, announced that it stopped recruitment in the hydroxychloroquine arm because the drug did not show any benefit [
[23]
], consistent with findings of no benefit of hydroxychloroquine in a recent observational study in patients with COVID-19 reporting [[24]
], and no preventative effects of the drugs in a randomized controlled trial [[25]
]. In the RECOVERY study, there was a trend for increased mortality for patients treated with hydroxychloroquine: among 1,542 patients randomized to hydroxychloroquine for 28 days, the mortality was 25.7% compared with 23.5% in 3,132 patients randomized to usual care alone (hazard ratio 1.11 [95% confidence interval (CI) 0.98–1.26]; P = 0.10). Neither was any benefit observed for any other outcome. A perusal of the COVID-evidence database (https://covid-evidence.org/database) shows over 100 trials involving hydroxychloroquine alone or in various combinations and comparisons. Recruitment in these trials may become more difficult after the release of the RECOVERY results. Many of them may stop or become futile even if they do not officially stop recruitment.5. Contributing data for rapid cross-trial international meta-analyses
A cross-trial analysis contributing promptly the already collected data from existing ongoing (hydroxy)chloroquine trials would be most informative in appraising the current situation and deciding both about the fate of these trials and the fate of these antimalarial drugs for COVID-19 in general [
[26]
]. Cross-sharing of preliminary data within trial teams and Data and Safety Monitoring Board (DSMB) is a practice that has been used often even in the pre-COVID-19 era. Typically, it has been limited to more narrow research agendas and a more limited number of trials. However, in the COVID-19 pandemic, it can become more routine and systematic, especially when major decisions need to be taken about the research agenda at large and its implications. Leveraging the analytic power of peers to exploit a maximum of information from available clinical data may be essential to adequately inform medical and political stakeholders, particularly when institutionalized mechanisms of quality control are limited under major time constraints such as in the current pandemic and when the external evidence evolves in so unpredictable ways. A collaboration of trialists and metaresearchers may achieve the best quality of such endeavors to collect and synthesize available evidence as quickly and as rigorously as possible.As of the writing of this commentary, a cross-trial international collaborative analysis of randomized trials of hydroxychloroquine and chloroquine has already been performed and released as a preprint [
[26]
]. Sixty-three registered trials were deemed potentially eligible, and all-cause mortality data were retrievable for 28 of them, including 14 unpublished trials (1,308 patients) and 14 publications/preprints (9,011 patients). The combined summary odds ratio on all-cause mortality for hydroxychloroquine was 1.11 (95% CI: 0.1, 1.20) and for chloroquine was 1.77 (95% CI: 0.15, 21.13). As additional trials may be willing to contribute data in the future, whatever residual uncertainty may decrease even further. A similar international collaborative meta-analysis has been launched trying to retrieve and combine data from dozens of randomized trials of convalescent plasma, and the concept may be extended to all interventions where the evidence is fragmented across large numbers of trials, many of which may be difficult to publish because of perceived futility.6. Toward implementation of sharing patient-level data from clinical trials
Open synthesis facilitating the conduct of living (continuously updated) and rapid systematic reviews and meta-analysis should become the norm in the COVID-19 era [
[9]
]. Flexible sharing between investigator teams and DSMB groups can happen without the need of engaging public data sharing platforms. DSMBs can be facilitators of efficient sharing in the COVID-19 era [[27]
]. However, when it comes to wider public data sharing, platforms dedicated for sharing patient-level clinical data such as the Yale University Open Data Access Project (https://yoda.yale.edu/) and clinicaldatarequest.com have been already established [[28]
,[29]
]. These platforms encourage the view that deidentified participant-level data can be shared in a transparent way that guards the privacy of the patients, interests of the original investigators, but multiplies the value of data for scientific merits to accelerate drug development. COVID-19 has accelerated the use of other open science resources, such as preprint servers like medRxiv. It could also serve as a catalyst for the wider use of platforms for sharing of participant-level data.7. Sharing of data from observational studies to identify risk profiles and predictors of treatment success
Finally, the principles of sharing patient-level data can be extended also to observational data sets and participatory research [
[30]
]. Nonrandomized data should be used very cautiously for making inferences about treatment effects in the COVID-19 crisis. Such data can offer complementary evidence about real-world use, and they may offer hints for leads that could then be pursued in randomized trials. Availability of individual-level data would be useful even for addressing fundamental questions of the epidemiology of the COVID-19 pandemic. For example, currently available situational reports from different countries and locations present very heterogeneous information on the demographic profile of COVID-19 deaths and comorbidities [[31]
]. Open access to large-scale patient-level data sets could help to identify important factors and modulating variables for identifying risk profiles and predicting treatment success. Having said that, we caution that such observational studies often lack prior consent from patients, or include highly individualized data sets such as generated from electronic health records, which may render the anonymization of the data more difficult. Creative solutions can be adopted in most settings, for example, patient-level data may be restricted to key variables that allow proper deanonymization. Alternatively, aggregated data may be shared for meta-analytical studies, which would still allow building detailed risk profiles fully accounting for variability across different countries and disease epicenters.8. Conclusions
Overall, in view of the speed of action required to fight the current pandemic, we strongly believe that it is time that sharing clinical trial and other patient-relevant data among the medical research community needs to become a flexible and realistic default option. Open science principles may become a great asset in the fight against COVID-19 and beyond.
References
- FDA guidance on conduct of clinical trials on medical products in the COVID-19 crisis.in: Services USDoHaH, Administration FaD. FDA, Rockville, MD, USA2020
- Randomized clinical trials and COVID-19: managing expectations.JAMA. 2020; 323: 2262-2263
- Coronavirus (COVID-19) update: FDA revokes emergency use authorization for chloroquine and hydroxychloroquine.FDA, Rockville, MD, USA2020
- Open access to clinical trials data.JAMA. 2014; 312: 1002-1003
- Dexamethasone in hospitalized patients with Covid-19 - preliminary report.N Engl J Med. 2020; (Epub ahead of print)
- Evaluation of evidence of statistical support and corroboration of subgroup claims in randomized clinical trials.JAMA Intern Med. 2017; 177: 554-560
- Association between administration of systemic corticosteroids and mortality among critically ill patients with COVID-19: a meta-analysis.JAMA. 2020; 324: 1330-1341
- A large number of COVID-19 interventional clinical trials were registered soon after the pandemic onset: a descriptive analysis.J Clin Epidemiol. 2020; 125: 170-178
- Open synthesis and the coronavirus pandemic in 2020.J Clin Epidemiol. 2020; 126: 184-191
- Data sharing statements for clinical trials: a requirement of the International Committee of Medical Journal.Lancet. 2017; 389: e12-e14
- Sharing clinical trial data: a proposal from the international Committee of medical Journal.Lancet. 2016; 387: e9-e11
- UK publicly funded Clinical Trials Units supported a controlled access approach to share individual participant data but highlighted concerns.J Clin Epidemiol. 2016; 70: 17-25
- Evaluation of clinical trial data sharing policy in leading medical journals.MedRxiv. 2020;
- Data anonymisation - a key enabler for clinical data sharing.2017 (Available at)https://www.ema.europa.eu/en/documents/report/report-data-anonymisation-key-enabler-clinical-data-sharing_en.pdfDate accessed: April 16, 2020
- Clinical trial participants' views of the risks and benefits of data sharing.N Engl J Med. 2018; 378: 2202-2211
- Use of hydroxychloroquine and chloroquine during the COVID-19 pandemic: what every Clinician should Know.Ann Intern Med. 2020; 172: 754-755
- Hydroxychloroquine or chloroquine with or without a macrolide for treatment of COVID-19: a multinational registry analysis.Lancet. 2020;
- Retraction—hydroxychloroquine or chloroquine with or without a macrolide for treatment of COVID-19: a multinational registry analysis.Lancet. 2020; (published online)
- An open letter to Mehra et al. and The Lancet.Zenodo. 2020;
- Expression of concern: hydroxychloroquine or chloroquine with or without a macrolide for treatment of COVID-19: a multinational registry analysis.Lancet. 2020; 395 (published online)
- Retraction: Cardiovascular disease, drug Therapy, and mortality in Covid-19.N Engl J Med. 2020; ([published online])https://doi.org/10.1056/NEJMoa2007621
- Learning from a retraction.Lancet. 2020; 396: 1056
- No clinical benefit from use of hydroxychloroquine in hospitalised patients with COVID-19.FDA, Rockville, MD, USA2020
- Observational study of hydroxychloroquine in hospitalized patients with Covid-19.N Engl J Med. 2020; 382: 2411-2418
- A randomized trial of hydroxychloroquine as postexposure prophylaxis for Covid-19.N Engl J Med. 2020; 383: 517-525
- Mortality outcomes with hydroxychloroquine and chloroquine in COVID-19: an international collaborative meta-analysis of randomized trials.medRxiv. 2020;
- Achieving effective informed oversight by DMCs in COVID clinical trials.J Clin Epidemiol. 2020; 126: 167-171
- Open access platforms for sharing clinical trial data.JAMA. 2016; 316: 666
- The Yale open data access (YODA) project--A mechanism for data sharing.N Engl J Med. 2016; 375: 403-405
- COllaborative open platform E-cohorts for research acceleration in trials and epidemiology.J Clin Epidemiol. 2020; 124: 139-148
- Population-level COVID-19 mortality risk for non-elderly individuals overall and for non-elderly individuals without underlying diseases in pandemic epicenters.Environ Res. 2020; 188: 109890
Article info
Publication history
Published online: October 13, 2020
Accepted:
October 10,
2020
Footnotes
Conflict of interest: None of the authors declares a conflict of interest.
Identification
Copyright
© 2020 Published by Elsevier Inc.
