If you don't remember your password, you can reset it by entering your email address and clicking the Reset Password button. You will then receive an email that contains a secure link for resetting your password
If the address matches a valid account an email will be sent to __email__ with instructions for resetting your password
We have proposed that three scientific criteria are important for the ethical justification of randomized clinical trials (RCTs): (1) they should be designed around a clear hypothesis; (2) uncertainty should exist around that hypothesis; (3) that uncertainty should be as established through a systematic review. We hypothesized that the majority of a sample of recently published RCTs would not explicitly incorporate these criteria, therefore rendering them potentially unjustified on scientific grounds.
Study Design and Setting
Cross-sectional analysis of all RCTs published in the New England Journal of Medicine and the Journal of the American Medical Association in 2015. Each article and protocol was reviewed for: (1) a clearly stated central hypothesis; (2) references to “equipoise,” or “consensus;” (3) some indication of evidentiary uncertainty; (4) a meta-analysis or systematic review surrounding the hypothesis or study question.
We included 208 RCT articles and 199 protocols. Among combined articles and protocols, 76% had a clearly stated hypothesis, 99% referenced some form of uncertainty, and 54% cited a relevant systematic review or meta-analysis. Only 44% of combined texts contained all three scientific criteria. There were few references to “equipoise” (10%) or “consensus” (11%), and those references to equipoise were most often inconsistent with accepted definitions.
The majority of RCTs (56%) did not meet the three scientific criteria described previously and therefore may be scientifically and therefore ethically unjustified. We recommend that “equipoise,” “clinical equipoise,” and “lack of consensus” be abandoned as scientific criteria for RCTs and be replaced by an expectation that RCTs have a clearly stated, meaningful hypothesis around which uncertainty has been established through a systematic review of the literature.
The majority of randomized clinical trials (RCTs) (56%) published in two prominent journals do not fulfill a set of 3 scientific criteria that we have proposed as necessary for the ethical justification of RCTs. As a consequence, many RCTs may be scientifically unjustified and potentially unethical as well.
What this adds to what was known?
For randomized clinical trials (RCTs) to be ethically justified, they must be scientifically justified. How RCTs should be scientifically justified is unclear.
We have proposed at least three scientific criteria that are important for this process: (1) they should be designed around a clear hypothesis; (2) uncertainty should exist around that hypothesis; (3) that uncertainty should be as established through a systematic review of available knowledge.
What is the implication and what should change now?
Despite being cited as the standard for justifying an RCT, “equipoise” and “clinical equipoise” are rarely mentioned in published RCTs. The terms are used inconsistently and often inaccurately. They should be abandoned.
RCTs should be required to have a clear hypothesis and to document the presence of uncertainty around the research question through a systematic review or meta-analysis.
Randomized clinical trials (RCTs) are essential for the acquisition of new medical knowledge, but they are not without risks to patients, and they are also associated with significant costs [
]. Therefore, researchers, regulators, funding agencies, and publishers should be able to determine when an RCT is ethically justified at least in virtue of its scientific potential, meaning when it is necessary over other forms of research, or over no research at all. We acknowledge that there are other important factors—such as informed consent—that also contribute to the ethical justification for an RCT, which we do not address here.
When as much as 85% of clinical research may be wasteful [
At present, no universally accepted criteria exist to guide stakeholders in determining when an RCT is ethically justified, at least in virtue of its scientific quality. While Consolidated Standards of Reporting Trials (CONSORT) guidelines require that published RCTs report their rationales, they do not elaborate how to go about evaluating a rationale [
] (Table 1). First, an RCT should address a clearly stated question and test a specific hypothesis. Second, the RCT's central question should not already have been answered, meaning that uncertainty should persist [
]. In other words, the research should not be redundant. Third, this evidentiary uncertainty should be established in a sufficiently rigorous fashion, for example, through a systematic review of available knowledge [
Would recently published RCTs meet our three criteria? To our knowledge, few studies have assessed RCTs for the elements proposed previously, and none has assessed them as a set of ethically relevant criteria. The lack of backgrounds and objectives in published RCTs has been highlighted as an important problem in a recent Cochrane review [
We hypothesized that the majority of the recent RCTs published in two high-impact journals—the New England Journal of Medicine (NEJM) and the Journal of the American Medical Association (JAMA)—would not satisfy the three criteria described previously.
We performed a cross-sectional review of all RCTs published in the NEJM and JAMA in the 2015 calendar year. We included studies that involved the randomization of participants to one of several treatment arms, the outcomes of which were compared. Articles were searched for inclusion if they were identified as “original articles” or “special articles” in NEJM, and as “original investigations” or “preliminary investigations” in JAMA. These two journals were selected because of their positions as well-regarded and high-impact journals and ease of access through our institution's subscriptions.
We searched each article and its research protocol (when available) for (1) a clearly stated central hypothesis, (2) references to “equipoise,” “clinical equipoise,” or “lack of consensus,” (3) some indication of evidentiary uncertainty (i.e., lack of evidence or evidence from only early-stage trials, conflicting evidence, or poor quality evidence), and (4) a meta-analysis or systematic review surrounding the hypothesis or study question. Across all categories, references were assessed for relevance to the trial's specific clinical question. We defined a hypothesis as a statement that expressed a specific prediction of an outcome. The hypothesis could be narrative or statistical in format, but we did not take sample size calculations, references to null hypotheses (without references to the alternate hypothesis), or trial designs (i.e., superiority, noninferiority, or equivalence) to be sufficient. Uncertainty was defined by any mention of the word uncertainty, of a lack of evidence, of poor quality evidence, or of conflicting evidence or guidelines. References to equipoise or consensus were determined by searching the body of the documents for those words. Citations of systematic reviews or meta-analyses were only deemed relevant if they were related to the clinical condition being studied and to the intervention or treatment being assessed in the experimental arm. We chose to search for meta-analyses, systematic reviews, and evidentiary uncertainty in the introduction and background sections of the articles, as CONSORT guidelines expect that past research and the rationale for the trial should be presented there [
A primary reviewer screened all abstracts of every published article to identify the RCTs and analyzed their full-text articles and protocols for the categories described previously. A second reviewer verified 10% of the articles and protocols at random, to determine the validity of the initial screening. Disagreements were resolved by consensus. Descriptive statistics were calculated using an Excel database.
We screened 406 records and included 208 articles that presented the results of original RCTs (Fig. 1). Of these, 199 had associated research protocols, which were also reviewed. There was 95% concordance between the two reviewers' assessments as to the presence of the categories of elements listed previously.
Among all sources searched (Fig. 2), 76% of the RCTs had a clearly stated hypothesis (51 in the article and 153 in the protocol, 158 in either), 99% referenced some form of evidentiary uncertainty (202 in the article, 188 in the protocol, 207 in either), and 54% related that uncertainty to the results of a systematic review or meta-analysis (87 in the article, 80 in the protocol, 112 in either).
References to “equipoise” (10%) or lack of consensus (11%) were rare, mostly occurring in the trials' protocols and not the manuscript itself (Fig. 2). The word “equipoise” appeared only 27 times in total, in association with only 22 trials (Table 2, full table available online on the journal's web site at www.elsevier.com). Moreover, analysis of its usages demonstrated how “equipoise” was defined in a number of different ways, including in relation to individual investigators' uncertainty (two mentions), to community disagreement established through surveys (two mentions), to variations in clinical practice (five mentions), to perceived weakness of the available evidence (six mentions), or to nonspecific comments about uncertainty (three mentions).
Table 2Selected usages of equipoise in relation to accepted definitions
Equipoise defined by…
It is unclear what the optimal ratio should be, and several leading centers have described good outcomes with both higher and lower ratios, confirming the presence of clinical equipoise for the proposed study groups.
Variations in clinical practice
The PIs of the Level 1 trauma centers selected for [the trial] unanimously declared equipoise
…where this reflects an individual surgeon's equipoise
There is no established Level I body of evidence… A critical level of clinical equipoise has been reached
Lack of RCT data
…training and study description videos are designed to present the two options with equipoise.
Lack of bias in the presentation of treatments
Vertebrobasilar occlusions are excluded because of the particularly dismal natural history of untreated basilar occlusion. Therefore, it is unclear whether clinical equipoise exists.
Poor natural history of condition
We found that the addition of sitagliptin to usual care among patients with glycemic equipoise did not affect the rates of major atherosclerotic events.
We believe there is true equipoise as to whether the intervention is beneficial [because] we conducted a clinician survey…
This is an open-label trial. However, in order to maintain equipoise, the study team… will not have access to the randomization codes
Blinding of investigators
Given the possibility of the “loss of equipoise” in an active procedural trial, it is critical to get recruitment completed fast and to maintain study-wide discipline in subject enrollment
Something that can be lost at a particular site
With respect to equipoise about the main research question there is: evidence from surveys of diverse practice between clinicians and centers; evidence from RCTs of net benefit to patients having other major surgery; evidence from observational studies…
Freedman's definition (surveys documenting variations), evidence from RCTs, and observational studies
Finally, a minority of RCTs (91, or 44%) met all of the elements through either their main articles (32) or protocols (61): (1) a clear hypothesis; (2) any uncertainty as reflected by “equipoise,” “lack of consensus,” or some discussion of the literature; and (3) a relevant meta-analysis or systematic review (Fig. 3).
According to our study, most of the RCTs published in two leading journals in 2015 did not meet a set of basic scientific criteria that could be used to ethically justify conducting an RCT, namely having a clear hypothesis, and establishing uncertainty through a systematic review of the literature. This is worrisome, because a trial that fails to meet these criteria may be unnecessary, harmful to its participants, and—because the ethical appropriateness of an RCT is conditional upon its scientific justification—even unethical.
Hypotheses are generally considered to be a necessary part of scientific research, including RCTs, as reflected in the CONSORT statement checklist [
]. Our results demonstrate that nearly one in four of the RCTs we reviewed did not have an explicitly stated hypothesis, and many of the hypotheses that were made explicit were scientifically trivial. For example, the Treatment of Sleep-Disordered Breathing with Predominant Central Sleep Apnea by Adaptive Servo Ventilation in Patients with Heart Failure trial, which evaluated adaptive servo-ventilation for central sleep apnea in systolic heart failure, presented the hypothesis that “the time to first event in the intervention group is different to the time to first event in the control group.” [
Just as in all other areas of scientific inquiry, hypotheses in medicine should be scientifically plausible and explicit, as they demarcate the scope of the RCT and dictate the necessary methods and statistical analyses; without them, the null hypothesis may be inappropriately rejected [
]. If there is insufficient knowledge to allow the researchers to speculate about the outcome of an RCT, the trial may be insufficiently justified on ethical grounds, and perhaps other forms of scientific research would be better suited to address the question.
4.2 Equipoise, consensus, and evidentiary uncertainty
Physicians and researchers generally seem to feel that uncertainty should exist before enrolling patients into an RCT and that conducting a trial when one treatment is known to be superior to its comparator is unethical. How such uncertainty should be established remains unclear. “Equipoise” is often invoked to reflect this standard of uncertainty, though multiple definitions appear in the literature. As initially defined by American legal scholar Charles Fried in 1974, “equipoise” necessitates that individual physicians enrolling patients into trials lack a preference for either intervention being tested, though no basis for that opinion was specified [
]. In 1987, bioethicist Benjamin Freedman proposed that individual physicians need not be undecided for a trial to be ethically acceptable—rather, disagreement at the community level created “clinical equipoise” and this is what justified conducting an RCT [
In our analysis of published RCTs, references to uncertainty were nearly universal, though “equipoise” or “clinical equipoise” appeared sparingly. Importantly, few references aligned with either acknowledged definition: two references reflected Fried's individual equipoise and two referenced formal surveys of a community of experts that would satisfy Freedman's “clinical equipoise” (Table 2). Several authors implied that equipoise existed whenever there was a variation in clinical practice or if an RCT had not yet been performed on the topic in question. Clear misapplications of equipoise were seen in references to patients' glycemic status or to the content of instructional resources. Given that equipoise is rarely mentioned and would appear to be poorly understood by researchers, its utility as an ethical standard in the justification of RCTs should be questioned.
4.3 Systematic reviews and meta-analyses
Nearly every study mentioned some form of evidentiary uncertainty, by which we mean uncertainty arising from a lack of scientific evidence surrounding the research question, due to poor quality evidence or due to conflicting evidence or guidelines. However, the authors of the studies did not necessarily establish how rigorously or systematically they had reviewed the literature and may have presented a biased selection of articles to justify their RCTs [
]. Ideally, the presence of evidentiary uncertainty surrounding a research question should be established by conducting a systematic literature review or meta-analysis, though this was only done in about one-half of the RCTs we reviewed. Systematic reviews and meta-analyses are generally considered to be the most effective ways of summarizing knowledge and assessing uncertainty around a potential research question [
], and our selection of RCTs, published exclusively in high-quality journals, likely reflects general practices. Second, our study methodology did not allow us to conclusively differentiate between problems of scientific validity and those of research reporting. For example, the trialists who authored these RCTs may have sought to establish equipoise but did not mention it or may have known of systematic reviews but did not cite them. Both of these would seem unlikely [
] and speak to the importance of these elements (or lack thereof) in the justification of RCTs.
RCTs should only proceed if they are scientifically and ethically justified. In this review of 208 RCTs, only 44% fulfilled a set of basic scientific criteria that would appear important to their ethical justification: stating a clear and meaningful hypothesis and documenting uncertainty around that hypothesis based on a systematic review. Therefore, as many of 56% of published RCTs reviewed in this study may not be (scientifically and hence) ethically justified. In our sample, the concept of equipoise was rarely mentioned and was almost never used in a manner consistent with accepted definitions. Therefore, “equipoise” would appear to fail as a standard according to which RCTs are scientifically justified.
We recommend that “equipoise,” “clinical equipoise,” and “lack of consensus” be abandoned as concepts used to determine the scientific warrant of RCTs as they lack standard and empirically stable meanings. We echo recent calls that funding agencies, regulators, research ethics boards, and publishers agree that a systematic review be required as part of the justification for RCT design [
], and we suggest that studies should also be required to contain explicit and meaningful hypotheses. Additional ethical criteria (such as consent) and methodological criteria (such as an appropriately defined population, sample size, and recruitment techniques) are also likely necessary for an RCT to be scientifically justified, though we did not assess these parameters in the present study. We would also encourage researchers to consider whether their research questions may be better addressed through other forms of clinical research that present fewer ethical risks to patients than do RCTs.
In 21st century medical research, randomized controlled trials (RCTs) are considered to generate the most useful form of medical knowledge . This is because they are perceived to be a uniquely efficient and reliable way of uncovering causal knowledge about very complex systems and processes. However, the centrality of RCTs to modern medical research does not guarantee that all such trials will, therefore, be ethically justified, nor that RCTs are always required to answer any meaningful clinical research question.