A pragmatic–explanatory continuum indicator summary (PRECIS): a tool to help trial designers

Article Outline

• Abstract
• 1. The problem
• 2. Ten ways in which pragmatic and explanatory trials can differ
• 3. Methods
• 4. The domains in detail
  • 4.1. Participant eligibility criteria
  • 4.2. Experimental intervention flexibility
  • 4.3. Experimental intervention practitioner expertise
  • 4.4. Comparison intervention
  • 4.5. Comparison intervention practitioner expertise
  • 4.6. Follow-up intensity
  • 4.7. Primary trial outcome
  • 4.8. Participant compliance with “prescribed” intervention
  • 4.9. Practitioner adherence to study protocol
  • 4.10. Analysis of the primary outcome
• 5. Examples
• 6. Comment
• Acknowledgments
• References
• Copyright

Abstract 

Objective

To propose a tool to assist trialists in making design decisions that are consistent with their trial's stated purpose.

Study Design and Setting

Randomized trials have been broadly categorized as either having a pragmatic or explanatory attitude. Pragmatic trials seek to answer the question, “Does this intervention work under usual conditions?,” whereas explanatory trials are focused on the question, “Can this intervention work under ideal conditions?” Design decisions make a trial more (or less) pragmatic or explanatory, but no tool currently exists to help researchers make the best decisions possible in accordance with their trial's primary goal. During the course of two international meetings, participants with experience in clinical care, research commissioning, health care financing, trial methodology, and reporting defined and refined aspects of trial design that distinguish pragmatic attitudes from explanatory.

Results

We have developed a tool (called PRECIS) with 10 key domains and which identifies criteria to help researchers determine how pragmatic or explanatory their trial is. The assessment is summarized graphically.

Conclusion

We believe that PRECIS is a useful first step toward a tool that can help trialists to ensure that their design decisions are consistent with the stated purpose of the trial.

Keywords: Randomized controlled trials, Clinical trial methodology, Pragmatic trial, Explanatory trial, Trial design, Clinical trial

Back to Article Outline

1. The problem 

Randomized trials have traditionally been broadly categorized as either an effectiveness trial or an efficacy trial, although we prefer the terms pragmatic and explanatory. Schwartz and Lellouch describe these two approaches toward clinical trials [1]. These authors coined the terms “pragmatic” to describe trials that help users choose between options for care, and “explanatory” to describe trials that test causal research hypotheses (i.e., that a given intervention causes a particular benefit).

We take the view that, in general, pragmatic trials are primarily designed to determine the effects of an intervention under the usual conditions in which it will be applied, whereas explanatory trials are primarily designed to determine the effects of an intervention under ideal circumstances [2]. Thus, these terms refer to a trial's purpose, and in turn, structure. The degree to which this purpose is met depends on decisions about how the trial is designed and, ultimately, conducted.

Very few trials are purely pragmatic or explanatory. For example, in an otherwise explanatory trial, there may be some aspect of the intervention that is beyond the investigator's control. Similarly, the act of conducting an otherwise pragmatic trial may impose some control resulting in the setting being not quite usual. For example, the very act of collecting data required for a trial that would not otherwise be collected in usual practice could be a sufficient trigger to modify participant behavior in unanticipated ways. Furthermore, several aspects of a trial are relevant, relating to choices of trial participants, health care practitioners, interventions, adherence to protocol, and analysis. Thus, we are left with a multidimensional continuum, rather than a dichotomy and a particular trial may display varying levels of pragmatism across these dimensions.

In this article, we describe an effort to develop a tool to assess and display the position of any given trial within the pragmatic–explanatory continuum. The primary aim of this tool is to help trialists to assess the degree to which design decisions align with the trial's stated purpose (decision-making vs. explanation). Our tool differs therefore from that of Gartlehner et al. [3] in that it is intended to inform trial design rather than provide a method of classifying trials for the purpose of systematic reviews. It can, however, also be used by research funders, ethics committees, trial registers, and journal editors to make the same assessment, provided trialists declare their intended purpose and adequately report their design decisions. Hence, reporting of pragmatic trials is addressed elsewhere [4].

Back to Article Outline

2. Ten ways in which pragmatic and explanatory trials can differ 

Trialists need to make design decisions in 10 domains that determine the extent to which a trial is pragmatic or explanatory. Explanatory randomized trials that seek to answer the question, “Can this intervention work under ideal conditions?” address these 10 domains with a view to maximizing whatever favorable effects an intervention might possess [2]. Table 1 illustrates how an explanatory trial, in its most extreme form, might approach these 10 domains.

Table 1. PRECIS domains illustrating the extremes of explanatory and pragmatic approaches to each domain

PRECIS, pragmatic–explanatory continuum indicator summary.

Pragmatic randomized trials that seek to answer the question, “Does this intervention work under usual conditions?” [5], [6] address these 10 domains in different ways when there are important differences between usual and ideal conditions. Table 1 illustrates the most extreme pragmatic response to these domains.

The design choices for a trial intended to inform a research decision about the benefit of a new drug are likely to be more explanatory (reflecting ideal conditions). Those for a later trial of the same drug intended to inform practical decisions by clinicians or policymakers are likely to be more pragmatic (reflecting usual conditions). When planning their trial, trialists should consider whether a trial's design matches the needs of those who will use the results. A tool to locate trial design choices within the pragmatic–explanatory continuum could facilitate these design decisions, help to ensure that the choices that are made reflect the intended purpose of the trial, and help others to appraise the extent to which a trial is appropriately designed for its intended purpose.

Such a tool could, for example, expose potential inconsistencies, such as using intensive adherence monitoring and intervention (explanatory tactics) in a trial being designed to answer a more pragmatic question. Alternatively, a trial might include a wide range of participants and meaningfully assess the impact (pragmatic tactics), but evaluate an intervention that is enforced or tightly monitored (explanatory tactics) and thus not widely feasible. By supporting the identification of potential inconsistencies such as these, a pragmatic–explanatory indicator could improve the extent to which trial designs are fit for purpose by highlighting design choices that do not support the needs of the intended users of the trial's results. In this article we introduce such a tool.

The pragmatic–explanatory distinction comprises a continuous spectrum, not an either–or dichotomy of the extremes, as illustrated in Table 1. Moreover, it is probably impossible ever to perform a “purely” explanatory or “purely” pragmatic trial. For example, no patients are perpetually compliant, and the hand of the most skilled surgeon occasionally slips, so there can never be a “pure” explanatory trial. Similarly, a “pure” pragmatic trial loses its purity as soon as its first eligible patient refuses to be randomized.

Back to Article Outline

3. Methods 

This PRECIS proposal was developed by an international group of interested trialists at two meetings in Toronto (2005 and 2008) and in the time between. The initiative grew from the (PRACTIHC) [7] project, a Canadian and European Union funded initiative to promote pragmatic trials in low- and middle-income countries.

The development of the indicator (which we have named “PRECIS” for “Pragmatic–Explanatory Continuum Indicator Summary”) began with the identification of key domains that distinguish pragmatic from explanatory trials. As illustrated in Table 1, they comprise:

During the 2005 meeting, eight domains emerged during a brainstorming session. Furthermore, five mutually exclusive definitions were used to assign the level of pragmatism in each domain. Attempts to use the initial tool on a number of published trials revealed some difficulties. The mutually exclusive categories were technically difficult to understand and use and in some cases contradictory among domains. The current approach, for the most part, is to consider a number of design tactics or restrictions consistent with an explanatory trial in each domain and the more tactics that are present, the more explanatory is the trial. However, these design tactics and restrictions (see “The domains in detail” section for some examples) are not equally important so it is not a simple matter of adding up tactics. Where exactly to place a trial on the pragmatic–explanatory continuum is, therefore, a judgment best made by trialists discussing these issues at the design stage of their trial and reaching consensus. Initially, the domains for intervention flexibility and practitioner expertise addressed both the experimental and comparison interventions. Discussions at the 2008 meeting led to the separation of experimental and comparison interventions into their own domains and the replacement of a domain regarding trial duration with the domain related to the nature of the primary outcome.

At this point, a brief explanation of our use of some terminology may be helpful. In this article, we view a trial participant as the recipient of the intervention. In many trials, the participants are patients. However, in a trial of a continuing education intervention, for example, the participants may be physicians. By practitioner we mean the people delivering the intervention. Again, for many trials the practitioners are physicians. For a continuing education intervention the practitioners may be trained instructors.

We defined the purpose of a pragmatic trial as answering the question, “Does an intervention work under usual conditions?” where we take “usual conditions” to mean the same as or very similar to the usual care setting. Characterizing the pragmatic extreme of each domain is less straight forward, because what is considered “usual care” may depend on context. For some interventions what is usual for each domain may vary across different settings. For example, the responsiveness and compliance of patients, adherence of practitioners to guidelines, and the training and experience of practitioners may be different in different settings. Thus, characterizing the pragmatic extreme requires specifying the settings for which a trial is intended to provide an answer. Occasionally a pragmatic trial addresses a question in a single specific setting. For example, a randomized trial of interventions to improve the use of active sick leave was designed to answer a pragmatic question under usual conditions specific to the Norwegian context, where active sick leave was being promoted as a public sickness benefit scheme offered to promote early return to modified work for temporarily disabled workers [8]. More often pragmatic trials will address questions across specific types of settings or across a wide range of settings. Examples of specific types of settings include settings where chloroquine-resistant falciparum malaria is endemic, where hospital facilities are in close proximity, or where trained specialists are available.

Conversely, we defined the purpose of an explanatory trial as answering the question, “Can an intervention work under ideal conditions?” Given this definition, characterizing the explanatory extreme of each domain is relatively straight forward and intuitive. It simply requires considering the design decisions one would make to maximize the chances of success. Thus, for example, one would select patients that are most likely to comply and respond to the intervention, ensure that the intervention is delivered in a way that optimizes its potential for beneficial effects, and ensure that it is delivered by well-trained and experienced practitioners.

Thus, we recommend that trialists or others assessing whether design decisions are fit for purpose do this in four steps:

For some trials there may not be any important difference between the pragmatic and explanatory extremes for some dimensions. For example, delivering an intervention, such as aspirin to someone with an acute myocardial infarction, does not require practitioner expertise. As mentioned earlier, for those domains where the extremes are clear, it should not be difficult to decide whether a design decision is at one extreme or the other. For design decisions that are somewhere in between the extremes, it can be more challenging to determine how pragmatic or explanatory a trial will be. It is for this reason that we recommend that all the members of the trial design team rate each domain and compare.

To facilitate steps three and four, we have identified a number of design tactics that either add restrictions typical of explanatory trials or remove restrictions in the fashion of pragmatism. The tactics that we describe below are not intended to prescriptive, exhaustive, or even ordered in a particular way, but illustrative. They are to aid trialists or others in assessing where within the pragmatic–explanatory continuum a domain is, allowing them to put a “tick” on a line representing the continuum. To display the “results” of this assessment, the lines for each domain are arranged like spokes of a wheel, with the explanatory pole near the “hub” and the pragmatic pole on the “rim.” The display is completed by joining the locations of all 10 indicators as we progress around the wheel.

The proposed scales seem to make sense intuitively and can be used without special training. Although we recognize alternative graphical displays are possible, we feel the proposed wheel plot is an appealing summary and is informative in at least three ways.

First, it depicts whether a trial is tending to take a broad view (as in a pragmatic trial asking whether an intervention does work, under usual conditions) or tending to be narrowly “focused” near the hub (as for an explanatory trial asking whether an intervention can work, under ideal conditions).

Second, it highlights inconsistencies in how the 10 dimensions will be managed in a trial. For example, if a trial is to admit all patients and practitioners (extremely pragmatic) yet will intensely monitor compliance and intervene when it falters (extremely explanatory), a single glance at the wheel will immediately identify this inconsistency. This allows the researcher to make adjustments, if possible and appropriate, in the design to obtain greater consistency with their objective in conducting the trial.

Third, it can help trialists better report any limitations in interpretation or generalization resulting from design inconsistencies. This could help users of the trial results to make better decisions.

Back to Article Outline

4. The domains in detail 

4.1. Participant eligibility criteria 

The most extremely pragmatic approach to eligibility would seek only to identify study participants with the condition of interest from as many sources (e.g., institutions) as possible. As one moves toward a more explanatory attitude, additional restrictions will be placed on the study population. These restrictions include the following:

The first three restrictions noted above are typically achieved by applying various exclusion criteria to filter out those participants thought least likely to respond to the intervention. So, explanatory trials tend to have more exclusion criteria than pragmatic trials. Exclusion criteria for known safety issues would not necessarily count against a pragmatic trial because such individuals would not be expected to get the intervention under usual practice.

4.2. Experimental intervention flexibility 

The pragmatic approach leaves the details of how to implement the experimental intervention up to the practitioners. For example, the details of how to perform a surgical procedure are left entirely to the surgeon, or how to deliver an educational program is left to the discretion of the educator. Additionally, the pragmatic approach would not dictate which co-interventions were permitted or how to deliver them. Several restrictions on the intervention's flexibility are possible.

4.3. Experimental intervention practitioner expertise 

A pragmatic approach would put the experimental intervention into the hands of all practitioners treating (educating, and others) the study participants. The practitioner choice can be restricted in a number of ways.

4.4. Comparison intervention 

Specification of the flexibility of the comparison intervention complements that of the experimental intervention flexibility domain. A pragmatic trial would typically compare an intervention to “usual practice” or the best available alternative management strategy, whereas an explanatory trial would restrict the flexibility of the comparison intervention and might, in the case of early-phase drug development trials, use a placebo rather than the best alternative management strategy as the comparator.

4.5. Comparison intervention practitioner expertise 

Similar comments apply as for the specification of the comparison intervention flexibility. In both cases the explanatory extreme would maximize the chances of detecting whatever benefits an intervention might have, whereas the pragmatic extreme would aim to find out the benefits and harms of the intervention in comparison with usual practice in the settings of interest.

4.6. Follow-up intensity 

The pragmatic position would be not to seek follow-up contact with the study participants in excess of the usual practice for the practitioner. The most extreme position is to have no contact with study participants and obtain outcome data by other means (e.g., administrative databases to determine mortality) instead. Various adjustments to follow-up intensity are possible and the extent to which these adjustments could lead to increased compliance or improved intervention response, follow-up intensity moves toward the explanatory end.

Often the required trial outcomes may be obtained only through contact with the participants. Even in the “no follow-up” approach, assessment of outcomes may be achieved with a single “end of study” follow-up. The end of study would need to be defined so that there is sufficient time for the desired study outcomes (see Primary trial outcome section) to be observed. When the follow-up is done in this way, it is unlikely to have an impact on compliance or responsiveness. However, there may often be considerable tension between unobtrusive follow-up and the ability to collect the necessary outcomes. It is often, although not always, the case that explanatory trials are interested in the effect of an intervention during the intervention period, or shortly after. On the other hand, pragmatic trials may follow patients well beyond the intervention period in their quest to answer the “does this work?” question. Such longer-term follow-up may well require more patient contact than usual care but is not necessarily inconsistent with a pragmatic approach if it does not result in patient management that differs from the usual conditions, which may in turn increase the chance of detecting an intervention effect beyond what would be expected under usual conditions.

4.7. Primary trial outcome 

For primary trial outcome, it is more intuitive to begin from the explanatory pole and describe the progression to the pragmatic pole. The most explanatory approach would consider a primary outcome (possibly surrogate as in dose-finding trials intended to demonstrate a biological response) that the experimental intervention is expected to have a direct effect on. Phase 3 and 4 trials often have patient-important outcomes and, thus, may be more pragmatic in this domain. There may well be central adjudication of the outcome or assessment of the outcome may require special training or tests not normally used to apply outcome definition criteria. Two obvious relaxations of the strict outcome assessment present in explanatory trials are the absence of central outcome adjudication and the reliance on usual training and measurement to determine the outcome status. For some interventions, the issue may be whether to only measure outcomes during the intervention period or up to a “reasonable” time after the intervention is complete. For example, stroke could be a primary outcome for explanatory and pragmatic trials. However, time-horizons may vary from: short-term following a one-time intervention (more explanatory) to long-term (more pragmatic).

4.8. Participant compliance with “prescribed” intervention 

The pragmatic approach recognizes that noncompliance with any intervention is a reality in routine medical practice. Because measurement of compliance may have the possibility of altering compliance, the pragmatic approach in a trial would be not to measure or use compliance information in any way. The more rigorous a trial is in measuring and responding to noncompliance of the study participants, the more explanatory it becomes:

For some trials, the goal of an intervention may be to improve compliance with a treatment guideline. Provided the compliance measurement is not used, directly or indirectly, to influence subsequent compliance, a trial could still be “very pragmatic” in this domain. On the other hand, if measuring compliance is part of the intervention (e.g., audit and feedback), this domain would, appropriately, move toward a more explanatory approach if audit and feedback could not be similarly applied as part of the intervention under usual circumstances.

4.9. Practitioner adherence to study protocol 

The pragmatic approach takes account of the fact that providers will vary in how they implement an intervention. A purely pragmatic approach therefore, would not be concerned with how practitioners vary or “customize” a trial protocol to suit their setting. By monitoring and (especially) acting on protocol nonadherence, a trial shifts toward being more explanatory:

4.10. Analysis of the primary outcome 

Recall that the pragmatic trial is concerned with the question, “Does the intervention work under usual conditions?” Assuming other aspects of a trial have been treated in a pragmatic fashion, an analysis that makes no special allowance for noncompliance, nonadherence, practice variability, and so on is most appropriate for this question. So, the pragmatic approach to the primary analysis would typically be an intention-to-treat analysis of an outcome of direct relevance to the study participants and the population they represent. The intention-to-treat analysis is also the norm for explanatory trials especially when regulatory approval for an intervention is being sought. However, there are various restrictions that may (additionally) be used to address the explanatory question, “Can this intervention work under ideal conditions?”:

For some explanatory trials (dose-finding trials are an example), it may be appropriate to have primary analysis restricted in the ways mentioned, otherwise such restricted analyses of the primary outcome would be preplanned as secondary analyses of the primary outcome. Note that if all domains of the trial were designed in an explanatory fashion and the trial was conducted accordingly, the above restrictions should have very little impact. A purely pragmatic approach would not consider these restricted analyses.

Back to Article Outline

5. Examples 

To demonstrate the use of the tool, we have applied the instrument to four trials exhibiting varying degrees of pragmatic and explanatory approaches. Table 2 describes how these trials addressed the 10 domains previously described. As we have stated previously, PRECIS is intended to be used at the design stage. We have applied it post hoc to these examples for illustrative purposes only.

Table 2. A PRECIS assessment of four trials (DOT [9], NASCET [10], CLASP [11], and Caritis et al. [12].)

Trial	Assessment of domain
Domain: participant eligibility criteria
DOT [9]	The trial admitted all-comers receiving care for newly diagnosed tuberculosis at two clinics. This was extremely pragmatic, but because only two clinics were studied and the setting of interest is (at a minimum) all of South Africa, it is not at the extreme edge.
NASCET [10]	NASCET enrollment was restricted to symptomatic patients stratified for carotid stenosis severity, with primary interest in a severe carotid stenosis (high-risk) group who were thought to be most likely to respond to endarterectomy, if it was efficacious. There was no prior compliance testing (other than a willingness to undergo angiography and several less invasive diagnostic tests). Exclusions included mental incompetence, another illness likely to cause death within 5yr, prior total stroke in the affected territory, a cardiac valvular or rhythm disorder (e.g., atrial fibrillation) likely to lead to embolic stroke, or prior carotid surgery on the affected artery. Patients also were temporarily ineligible if they had any of seven transient medical conditions (e.g., uncontrolled hypertension or diabetes, recent major surgery, unstable coronary heart disease). Thus, eligibility was very near the extreme explanatory end of the scale.
CLASP [11]	This trial had broad inclusion criteria (12–32 wk gestation at sufficient risk of pre-eclampsia or intrauterine growth retardation to consider aspirin (ASA) usage), few exclusion criteria and was conducted in a large (213) number of centers. This is extremely pragmatic.
Caritis et al. [12]	This trial recruited high-risk patients from 13 centers. Before patients were randomized, compliance was evaluated. Only those with 70% or better compliance were randomized. This is extremely explanatory in character.
Domain: experimental intervention flexibility
DOT [9]	The method of self-administration was left to the individual patient, who could delegate weekly drug collection visits to a family member. This was extremely pragmatic in character.
NASCET [10]	An endarterectomy had to be carried out (rather than stenting or some other operation), but the surgeon was given leeway in how it was performed (e.g., whether to use patches or temporary shunts). Also, simultaneous coronary artery bypass grafting was proscribed. Bilateral carotid endarterectomy could be performed provided the symptomatic side was operated on first. The same co-interventions (best medical care) were specified for both surgical and control patients. This was clearly very explanatory, but could be more so if the intraoperative procedures were also specified.
CLASP [11]	Patients were instructed to take one tablet/d unless their doctor advised otherwise. Compounds containing ASA were recommended against and a compound for analgesia was recommended. So, some flexibility (doctors opinion) was permitted. However, the medication recommendations are such that they would tend to maximize the difference between treatments. Thus, this domain is not completely pragmatic.
Caritis et al. [12]	Patients were instructed to take one tablet/d unless they were told that they had developed pre-eclampsia. They were given a list of medications to avoid and medication for analgesia. Because the criterion for stopping study drug is specified, this tends to be more explanatory in nature, although it is by no means extreme in that regard.
Domain: experimental intervention practitioner expertise
DOT [9]	All clinic nurses were involved, with no particular specialization or additional training. Patients were self-treating with no special training. Thus, an extremely pragmatic approach.
NASCET [10]	NASCET surgeons had to be approved by an expert panel, and were restricted to those who had performed at least 50 carotid endarterectomies in the last 24 months, with a postoperative complication rate (stroke or death within 30 d) of less than 6%. This was an extremely explanatory approach. All follow-up assessments were carried out by board-certified neurologists or their senior subspecialty trainees (a slightly less explanatory approach).
CLASP [11]	Patients remained under the care of their own doctors. This is the pragmatic approach.
Caritis et al. [12]	This is not explicitly stated in the trial report. However, we can make an educated guess. The patients were under the care of a physician at the participating center. Because this trial was studying high-risk patients, it is reasonable to assume that the participating centers were chosen because they have a relatively high volume of high-risk cases, which in turn suggests that specialists were involved in patient care rather than generalists. This tends to the more explanatory approach.
Domain: comparison intervention(s)
DOT [9]	Clinics already had the direct observation intervention in place, and this was not altered; extremely pragmatic.
NASCET [10]	In NASCET antiplatelet therapy (usually 1,300mg of ASA/d) was prescribed. Also, the co-interventions applied to surgical patients were also applied to control patients (antihypertensive therapy with blood pressure targets and feedback, antilipid and antidiabetic therapy) as indicated: an explanatory approach.
CLASP [11]	As both interventions were a simple tablet, this domain has been treated similar to the experimental arm.
Caritis et al. [12]	As both interventions were a simple tablet, this domain has been treated similar to the experimental arm.
Domain: comparison intervention(s) practitioner expertise
DOT [9]	All clinic nurses were involved, with no particular specialization or additional training, which was extremely pragmatic.
NASCET [10]	The patients in the medical arm were managed and followed by board-certified neurologists or their senior subspecialty trainees, just like the surgical patients.
CLASP [11]	Because there was no difference in care provider with respect to treatment, this domain has been treated similar to the experimental arm.
Caritis et al. [12]	Because there was no difference in care provider with respect to treatment, this domain has been treated similar to the experimental arm.
Domain: follow-up intensity
DOT [9]	No extra clinic visits were scheduled. In fact, in the experimental arm, no visits whatsoever were required because even the weekly drug collection could be delegated to a family member. This was the most extreme pragmatic approach.
NASCET [10]	NASCET patients had pre-scheduled appointments at 1, 3, 6, 9, 12, 16, 20, and 24 months (and every 4 months thereafter). Each consisted of a medical, neurologic, and functional-status assessment. All blood pressure records were reviewed centrally, and elevated readings triggered reminder letters. None of the 659 patients were lost to follow-up. A highly explanatory approach was taken here.
CLASP [11]	There was a single scheduled follow-up, which happened after delivery of the infant and any of the primary study outcomes. Infant deaths up to 1 yr were also recorded. This is very pragmatic.
Caritis et al. [12]	Study follow-ups were scheduled to occur with the standard patient care schedules at each center. Usually the patients were seen every 4 wk up to 28 wk gestation, then every 2 wk up to 36 wk gestation and then weekly thereafter until delivery. Although the visit schedule was no more intense than they would have seen at these centers outside the trial, there would have been trial-related data collected that may not normally have been done which may have altered patient management from standard. This is very explanatory but not extreme.
Domain: primary trial outcome
DOT [9]	The primary outcome was “successful treatment” which included all patients who were cured and all patients who completed the treatment. All patients were followed up for a year, until they completed their treatment, died, were classified as “incompletely treated,” or were lost to follow-up; very pragmatic.
NASCET [10]	The primary outcome was time to ipsilateral stroke, the clinically relevant, explanatory outcome most likely to be affected by carotid endarterectomy. Other outcomes were more pragmatic: all strokes, major strokes, and mortality were secondary outcomes.
CLASP [11]	The primary outcome of pre-eclampsia was defined in a clinically relevant way that required only investigations common to standard care. Deaths up to 1yr postdelivery were recorded and adjudicated for cause. This is very pragmatic, but not the most extreme position.
Caritis et al. [12]	The primary outcome of pre-eclampsia was defined in a clinically relevant way that required only investigations common to standard care. There was blinded adjudication of the primary outcome. There were a number of other short-term outcomes. Although the primary outcome itself is consistent with a pragmatic approach, the adjudication and focus on short-term outcomes moves this some ways toward an explanatory approach.
Participant compliance with “prescribed” intervention
DOT [9]	Compliance was an element of the outcomes, and so was measured for this purpose, but not used to improve patient compliance. This was pragmatic, but not at the most extreme end.
NASCET [10]	The experimental intervention in NASCET was offering a one-time operation. Because the 50% probability of operation was clearly stated in the original consent documents, patients who did not want surgery were unlikely to enter the trial (only 0.3% of admitted patients randomized to the operation refused it). This is a prophylactic strategy for achieving compliance and is thus, an explanatory approach.
CLASP [11]	Compliance was asked about at the follow-up visit. As this is after the completion of treatment, it could in no way affect compliance in the trial. Thus, it is extremely pragmatic.
Caritis et al. [12]	Compliance was measured by pill count and direct questioning during follow-up. A research nurse periodically contacted women to “survey and reinforce compliance.” This is an extremely explanatory approach.
Domain: practitioner adherence to study protocol
DOT [9]	There were no measurements of protocol adherence, and no adherence-improving strategies were used. This was the most pragmatic approach possible.
NASCET [10]	The completeness, timeliness, and accuracy of clinical data forms generated at admission, follow-up, and for events were monitored centrally. Both at regular intervals, and more frequently when they were deficient, the NASCET Principal Investigator made a personal visit to their center. In addition, blood pressure reports from each visit were scrutinized centrally, with letters pestering clinical collaborators when they were elevated. An extremely explanatory approach was evident here.
CLASP [11]	Not specified, assume not extreme in either direction.
Caritis et al. [12]	Not specified, assume not extreme in either direction.
Domain: analysis of primary outcome
DOT [9]	All randomized patients were included in the primary analysis. Patients who failed to meet the criteria for “successful treatment” (including those who died, were lost to follow-up, or transferred to another clinic) were classified “failures.” This was an extremely pragmatic approach.
NASCET [10]	The primary analysis was restricted to fatal and nonfatal strokes affecting the operated side of the cerebral circulation. In addition, blind adjudicators removed three NASCET patients after they were randomized because a review of their pre-randomization data revealed that they had other explanations for their symptoms (glaucoma, symptoms not arising from a carotid territory of the brain) or were inoperable (total occlusion of their carotid artery). However, patients were not excluded if they did not have a carotid endarterectomy or had uncontrolled blood pressure. This leaned toward an explanatory approach.
CLASP [11]	An intention-to-treat analysis was conducted on patients who completed the follow-up. Some subgroups, notably high-risk subgroups, were considered a priori. This is a fairly pragmatic approach.
Caritis et al. [12]	An intention-to-treat analysis was conducted on women with outcome data. An analysis, adjusted for compliance was also performed. A number of additional “explanatory” analyses were conducted. This is fairly explanatory in its approach.

PRECIS, pragmatic–explanatory continuum indicator summary; DOT, directly observed treatment of tuberculosis; NASCET, North American Symptomatic Carotid Endarterectomy Trial; CLASP, Collaborative Low-dose Aspirin Study in Pregnancy

The first example uses the trial of self-supervised and directly observed treatment of tuberculosis (DOT) [9]. The DOT trial asked the question: Among South African adults with newly diagnosed pulmonary tuberculosis, does five times weekly direct observation of pill swallowing by a nurse in the clinic, compared with self-administration, increase the probability that patients will take >80% of doses within 7 months of starting treatment, with no interruptions of >2 weeks? In this example, the “experimental” intervention was self-administration and the comparison intervention was DOT, which was widely used (throughout South Africa and elsewhere), but not adequately evaluated.

The second example uses the North American Symptomatic Carotid Endarterectomy Trial (NASCET) [10]. The NASCET trial asked the question: Among patients with symptomatic 70–99% stenosis of a carotid artery (and therefore at high risk of stroke), can the addition of carotid endarterectomy (performed by an expert vascular or neurosurgeon with an excellent track record) to best medical therapy, compared with best medical therapy alone, reduce the outcomes of major stroke or death over the next 2 years?

The third example uses the Collaborative Low-dose Aspirin Study in Pregnancy (CLASP) trial [11]. The CLASP trial, a placebo controlled trial, was designed to “provide reliable evidence about the overall safety of low-dose aspirin use in pregnancy and to find out whether treatment really produces worthwhile effects on morbidity and on fetal and neonatal mortality [11].”

The final example uses Caritis and colleagues' trial of low-dose aspirin to prevent pre-eclampsia in women at high risk [12]. This is another placebo controlled trial of aspirin designed to determine whether low-dose aspirin could reduce the incidence of pre-eclampsia among women at high risk for this condition.

Figure 1 shows a blank wheel graph for summarizing the indicators. All that is left is to mark each spoke to represent the location on the explanatory (hub) to pragmatic (“rim”) continuum and connect the dots.

• 
  • View Large Image
  • Download to PowerPoint

• Fig. 1 

  The blank pragmatic–explanatory continuum indicator summary (PRECIS) “wheel.”

Given the tactics used by the DOT trial in each of these dimensions, if we link each of them to their immediate neighbor as in Fig. 2a, we get a visual representation of the very “broad” pragmatic approach of this trial. Similarly, given the tactics used by the NASCET trial in each of these dimensions, Fig. 2b provides a visual representation of the, mostly, “narrow” explanatory approach of this trial. The final two examples are trials of the same intervention for the same condition. It can be seen from Fig. 2c and d that the CLASP trial tended to be more pragmatic than the Caritis trial.

• 
  • View Large Image
  • Download to PowerPoint

• Fig. 2 

  (a) PRECIS summary of a randomized controlled trial of self-supervised and directly observed treatment of tuberculosis (DOT) [9]. (b) PRECIS summary of the North American Symptomatic Carotid Endarterectomy Trial (NASCET) of carotid endarterectomy for symptomatic patients with high-grade carotid stenosis [10]. (c) PRECIS summary of a randomized trial of low-dose aspirin for the prevention and treatment of pre-eclampsia (Collaborative Low-dose Aspirin Study in Pregnancy [CLASP]) [11]. (d) PRECIS summary of a randomized trial of low-dose aspirin for the prevention of pre-eclampsia in women at high risk [12].

Back to Article Outline

6. Comment 

The PRECIS tool is an initial attempt to identify and quantify trial characteristics that distinguish between pragmatic and explanatory trials to assist researchers in designing trials. As such, we welcome suggestions for its further development. For example, the tool is applicable to individually randomized trials. It would probably apply to cluster randomized trials as well, but we have not tested it for those designs.

It is not hard to imagine that a judgment call is required to position the dots on the wheel diagram, especially in domains that are not at an extreme. Because trials are typically designed by a team of researchers, PRECIS should be used by all involved in the design of the trial, leading to a consensus view on where the trial is situated within the pragmatic–explanatory continuum. The possible subjectiveness of dot placement should help focus the researcher's attention on those domains that are not as pragmatic or explanatory as they would like. Clearly, those domains where consensus is difficult to achieve warrant more attention.

There are other characteristics that may more often be present in pragmatic trials, but because they can also be found in explanatory trials, they are not immediately helpful for discrimination. An appreciation of these characteristics help round out the picture somewhat and assist with the interpretation of a given trial. For example, in a pragmatic trial, the “control group” is, by definition, standard care. So, one would be unlikely to use a placebo group in a pragmatic trial. Therefore, whereas the presence of a placebo group suggests an explanatory trial, absence of a placebo group does not necessarily suggest a pragmatic trial. Another example of this is blinding, whether it be blinded intervention delivery or outcome assessment blinded to treatment assignment. Blinding is desirable in all trials to the extent possible. Blinding may be less practical to achieve in some pragmatic trials, but that does not imply that blinding is inconsistent with a pragmatic trial.

Understanding the context for the applicability of the trial results is essential for all trials. For example, the intervention studied in a pragmatic trial should be one that is feasible to implement in the “real world” after the completion of the trial. However, feasibility is often context specific. For example, an intervention could be easy to implement in Ontario, Canada, but all but impossible in a low-income country owing to cost, different health care delivery systems, and many other reasons.

Our initial experiences developing PRECIS suggest that it has the potential to be a useful tool for trial design, although we anticipate that some refinement of the scales will be required. The reporting of pragmatic trials is addressed elsewhere [4]. The simple graphical summary is a particularly appealing feature of this tool. We believe it to have value for the planning of trials and assessing whether the design of a trial is fit for purpose. It can help ensure the right balance is struck to achieve the primary purpose of a trial, which may be to answer an “explanatory” question about whether an intervention can work under ideal conditions or to answer a “pragmatic” question about whether an intervention does work under usual conditions. PRECIS highlights the multidimensional nature of the pragmatic–explanatory continuum. This multidimensional structure should be borne in mind by trial designers and end-users alike so that overly simplistic labeling of trials can be avoided.

We would also like to caution readers to not confound the structure of a trial with its usefulness to potential users. Schwartz and Lellouch clearly linked the ability of a trial to meet its purpose with decisions about how the trial is designed and that, taken together, these decisions affect where the trial is placed on the explanatory-pragmatic continuum [1]. However, how useful a trial is depends not only on design but on the similarity between the user's context and that of the trial. Although it is unreasonable to expect the results of a trial to apply in all contexts, trials should be designed and reported in such a way that users of the results can make meaningful judgments about applicability to their own context [13].

Finally, we stress that this article, building on earlier work from multiple investigators, describes a “work in progress.” We welcome suggestions from all who read it, including especially those who wish to join us in its further development. The words with which Daniel Schwartz and Joseph Lellouch closed their 1967 article continue to apply: “This article makes no pretention to originality, nor to the provision of solutions; we hope we have clarified certain issues to the extent of encouraging further discussion.”

Back to Article Outline

Acknowledgments 

The authors are especially indebted to Dr. David L. Sackett for his encouragement and advice during the development of the tool and preparation of this article. We also acknowledge the contributions made by the numerous attendees at the Toronto workshops in 2005 and 2008. The PRACTiHC group was supported by the European Commission's 5th Framework INCO program, contract ICA4-CT-2001-10019. The 2005 Toronto meeting was supported by the Canadian Institutes for Health Research grant number FRN 63095. The 2008 Toronto meeting was supported by the UK Medical Research Council, the Centre for Health Services Sciences at Sunnybrook Hospital, Toronto, Canada, the Centre for Medical Technology Policy, Baltimore, Maryland, USA, and the National Institute for Health and Clinical Excellence, UK.

Back to Article Outline

References 

1. Schwartz D, Lellouch J. Explanatory and pragmatic attitudes in therapeutical trials. J Chronic Dis. 1967;20:637–648[reprinted in this issue; J Clin Epidemiol 2009;62:498–504]
   • View In Article
   • MEDLINE
   • CrossRef
2. Sackett DL. Explanatory vs. management trials. In:  Haynes RB,  Sackett DL,  Guyatt GH,  Tugwell P editor. Clinical epidemiology: how to do clinical practice research. Philadelphia: Lippincott, Williams and Wilkins; 2006;
   • View In Article
3. Gartlehner G, Hansen RA, Nissman D, Lohr KN, Carey TS. A simple and valid tool distinguished efficacy from effectiveness studies. J Clin Epidemiol. 2006;59:1040–1048
   • View In Article
   • Abstract
   • Full Text
   • Full-Text PDF (147 KB)
   • CrossRef
4. Zwarenstein M, Treweek S, Gagnier J, Altman DG, Tunis S, Haynes B, et al. Improving the reporting of pragmatic trials: an extension of the CONSORT statement. BMJ 2008;337:a2390.
   • View In Article
5. Tunis SR, Stryer DB, Clancy CM. Practical clinical trials: Increasing the value of clinical research for decision making in clinical and health policy. JAMA. 2003;290:1624–1632
   • View In Article
   • CrossRef
6. Tunis SR. A clinical research strategy to support shared decision making. Health Aff. 2005;24:180–184
   • View In Article
7. Pragmatic Randomized Controlled Trials in HealthCare . http://www.practihc.org/
   • View In Article
8. Scheel IB, Hagen KB, Herrin J, Carling C, Oxman AD. Blind faith? The effects of promoting active sick leave for back pain patients. A cluster-randomized trial. Spine. 2002;27:2734–2740
   • View In Article
   • CrossRef
9. Zwarenstein M, Schoeman JH, Vundule C, Lombard CJ, Tatley M. Randomised controlled trial of self-supervised and directly observed treatment of tuberculosis. Lancet. 1998;352:1340–1343
   • View In Article
   • Abstract
   • Full Text
   • Full-Text PDF (71 KB)
   • CrossRef
10. North American Symptomatic Carotid Endarterectomy Trial Collaborators. Beneficial effect of carotid endarterectomy in symptomatic patients with high-grade carotid stenosis. N Engl J Med. 1991;325:445–453
    • View In Article
    • MEDLINE
    • CrossRef
11. CLASP (Collaborative Low-dose Aspirin Study in Pregnancy) Collaborative Group. CLASP: a randomized trial of low-dose aspirin for the prevention and treatment of pre-eclampsia among 9364 pregnant women. Lancet. 1994;343:619–629
    • View In Article
    • Abstract
    • CrossRef
12. Caritis S, Sibai B, Hauth J, Lindheimer MD, Klebanoff M, Thom E, et al. Low-dose aspirin to prevent preeclampsia in women at high risk. N Engl J Med. 1998;338:701–705
    • View In Article
    • MEDLINE
    • CrossRef
13. Rothwell PM. External validity of randomised controlled trials: “to whom do the results of this trial apply?. Lancet. 2005;365:82–93
    • View In Article
    • Abstract
    • Full Text
    • Full-Text PDF (134 KB)
    • CrossRef