Advertisement

An analysis reveals differences between pragmatic and explanatory diagnostic accuracy studies

  • Patrick M. Bossuyt
    Correspondence
    Corresponding author: Amsterdam University Medical Centers, Room J1B-214, PO Box 22660, 1100 DD Amsterdam, the Netherlands. Tel.: +31 20 566 3240; fax: +31 20 566 9004.
    Affiliations
    Department of Clinical Epidemiology, Biostatistics and Bioinformatics, Amsterdam Public Health Research Institute, Amsterdam University Medical Centers, Amsterdam, the Netherlands
    Search for articles by this author
  • Maria Olsen
    Affiliations
    Department of Clinical Epidemiology, Biostatistics and Bioinformatics, Amsterdam Public Health Research Institute, Amsterdam University Medical Centers, Amsterdam, the Netherlands
    Search for articles by this author
  • Chris Hyde
    Affiliations
    Exeter Test Group, Institute of Health Research, College of Medicine and Health, University of Exeter, Exeter UK
    Search for articles by this author
  • Jérémie F. Cohen
    Affiliations
    Inserm UMR 1153, Obstetrical, Perinatal and Pediatric Epidemiology Research Team, Research Centre for Epidemiology and Biostatistics Sorbonne Paris Cité (CRESS), Paris Descartes University, Paris, France

    Department of General Pediatrics and Pediatric Infectious Diseases, Necker - Enfants malades hospital, Assistance Publique - Hôpitaux de Paris, Paris Descartes University, Paris, France
    Search for articles by this author
Published:September 24, 2019DOI:https://doi.org/10.1016/j.jclinepi.2019.09.017

      Abstract

      Objectives

      The objective of this study was to clarify a difference between two approaches while evaluating the diagnostic accuracy of medical tests, labeled here as “pragmatic” vs. “explanatory” studies.

      Methods

      Using the definitions and characteristics described by Schwartz and Lellouch for randomized trials of interventions, and Schwartz' more general distinction between a pragmatic and an explanatory approach in medical research, we define a similar continuum for diagnostic accuracy studies. Explanatory studies aim to better understand the behavior of a test; pragmatic ones are done to support recommendations or decisions about using the test in clinical practice.

      Results

      Pragmatic test accuracy studies differ from explanatory test accuracy studies in several ways. The difference in aims has implications for key elements of study design, such as the study eligibility criteria, the recruitment of patients, the reference standard, and the choice of the statistical analysis. Explanatory accuracy studies are often designed to test a hypothesis. They are typically selective in recruitment, may include “healthy controls,” with a small sample size, often recruited at a single center. They ignore testing failures in the analysis and more often present their results as ROC curves. By contrast, pragmatic studies are designed to guide decision making. They ideally will recruit a single, large, and representative group of patients at multiple sites and will more often present their results as estimates of sensitivity and specificity or predictive values at a prespecified threshold.

      Conclusion

      Distinguishing between a pragmatic and an explanatory approach can help in the design, analysis, and interpretation of diagnostic accuracy studies. It can clarify debates about the appropriateness of design features to the study purpose and about the validity and applicability of study findings.

      Keywords

      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'

      Subscribe:

      Subscribe to Journal of Clinical Epidemiology
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect

      References

        • Schwartz D.
        • Lellouch J.
        Explanatory and pragmatic attitudes in therapeutical trials.
        J Chronic Dis. 1967; 20: 637-648
        • Schwartz D.
        • Lellouch J.
        Explanatory and pragmatic attitudes in therapeutical trials.
        J Clin Epidemiol. 2009; 62: 499-505
        • Knottnerus J.A.
        • Tugwell P.
        Research methods must find ways of accommodating clinical reality, not ignoring it: the need for pragmatic trials.
        J Clin Epidemiol. 2017; 88: 1-3
        • Koppenaal T.
        • Linmans J.
        • Knottnerus J.A.
        • Spigt M.
        Pragmatic vs. explanatory: an adaptation of the PRECIS tool helps to judge the applicability of systematic reviews for daily practice.
        J Clin Epidemiol. 2011; 64: 1095-1101
        • Zuidgeest M.G.P.
        • Goetz I.
        • Groenwold R.H.H.
        • Irving E.
        • van Thiel G.
        • Grobbee D.E.
        • et al.
        Series: pragmatic trials and real world evidence: paper 1. Introduction.
        J Clin Epidemiol. 2017; 88: 7-13
        • Nieuwenhuis J.B.
        • Irving E.
        • Oude Rengerink K.
        • Lloyd E.
        • Goetz I.
        • Grobbee D.E.
        • et al.
        Pragmatic trial design elements showed a different impact on trial interpretation and feasibility than explanatory elements.
        J Clin Epidemiol. 2016; 77: 95-100
        • Zwarenstein M.
        • Treweek S.
        • Loudon K.
        PRECIS-2 helps researchers design more applicable RCTs while CONSORT extension for pragmatic trials helps knowledge users decide whether to apply them.
        J Clin Epidemiol. 2017; 84: 27-29
        • Schwartz D.
        Conaissance ou décision. Attitude explicative - Attitude pragmatique.
        J de la société statistique de Paris. 1994; 135: 3-18
        • Knottnerus J.A.
        • van W.C.
        • Muris J.W.
        Evaluation of diagnostic procedures.
        BMJ. 2002; 324: 477-480
        • Bossuyt P.M.
        • Reitsma J.B.
        • Linnet K.
        • Moons K.G.
        Beyond diagnostic accuracy: the clinical utility of diagnostic tests.
        Clin Chem. 2012; 58: 1636-1643
        • Linnet K.
        • Bossuyt P.M.
        • Moons K.G.
        • Reitsma J.B.
        Quantifying the accuracy of a diagnostic test or marker.
        Clin Chem. 2012; 58: 1292-1301
        • Zwarenstein M.
        • Treweek S.
        What kind of randomized trials do we need?.
        J Clin Epidemiol. 2009; 62: 461-463
        • Oliveira S.B.
        • Monteiro I.M.
        Diagnosis and management of inflammatory bowel disease in children.
        BMJ. 2017; 357: j2083
        • Oyaert M.
        • Boel A.
        • Jacobs J.
        • Van den Bremt S.
        • De Sloovere M.
        • Vanpoucke H.
        • et al.
        Analytical performance and diagnostic accuracy of six different faecal calprotectin assays in inflammatory bowel disease.
        Clin Chem Lab Med. 2017; 55: 1564-1573
        • Dolwani S.
        • Metzner M.
        • Wassell J.J.
        • Yong A.
        • Hawthorne A.B.
        Diagnostic accuracy of faecal calprotectin estimation in prediction of abnormal small bowel radiology.
        Aliment Pharmacol Ther. 2004; 20: 615-621
        • Walker G.J.
        • Moore L.
        • Heerasing N.
        • Hendy P.
        • Perry M.H.
        • McDonald T.J.
        • et al.
        Faecal calprotectin effectively excludes inflammatory bowel disease in 789 symptomatic young adults with/without alarm symptoms: a prospective UK primary care cohort study.
        Aliment Pharmacol Ther. 2018; 47: 1103-1116
        • Fagerhol M.K.
        • Dale I.
        • Anderson T.
        Release and quantitation of a leucocyte derived protein (L1).
        Scand J Haematol. 1980; 24: 393-398
        • Røseth A.G.
        • Fagerhol M.K.
        • Aadland E.
        • Schjonsby H.
        Assessment of the neutrophil dominating protein calprotectin in feces. A methodologic study.
        Scand J Gastroenterol. 1992; 27: 793-798
        • Horvath A.R.
        • Lord S.J.
        • StJohn A.
        • Sandberg S.
        • Cobbaert C.M.
        • Lorenz S.
        • et al.
        From biomarkers to medical tests: the changing landscape of test evaluation.
        Clin Chim Acta. 2014; 427: 49-57
        • Lord S.J.
        • Staub L.P.
        • Bossuyt P.M.
        • Irwig L.M.
        Target practice: choosing target conditions for test accuracy studies that are relevant to clinical practice.
        BMJ. 2011; 343: d4684
        • Naaktgeboren C.A.
        • Bertens L.C.
        • van Smeden M.
        • de Groot J.A.
        • Moons K.G.
        • Reitsma J.B.
        Value of composite reference standards in diagnostic research.
        BMJ. 2013; 347: f5605
        • Rutjes A.W.
        • Reitsma J.B.
        • Vandenbroucke J.P.
        • Glas A.S.
        • Bossuyt P.M.
        Case-control and two-gate designs in diagnostic accuracy studies.
        Clin Chem. 2005; 51: 1335-1341
        • Bachmann L.M.
        • Puhan M.A.
        • ter Riet G.
        • Bossuyt P.M.
        Sample sizes of studies on diagnostic accuracy: literature survey.
        BMJ. 2006; 332: 1127-1129
        • Ochodo E.A.
        • de Haan M.C.
        • Reitsma J.B.
        • Hooft L.
        • Bossuyt P.M.
        • Leeflang M.M.
        Overinterpretation and misreporting of diagnostic accuracy studies: evidence of "spin".
        Radiology. 2013; 267: 581-588
        • Pepe M.S.
        • Janes H.
        • Li C.I.
        • Bossuyt P.M.
        • Feng Z.
        • Hilden J.
        Early-phase studies of biomarkers: what target sensitivity and specificity values might confer clinical utility?.
        Clin Chem. 2016; 62: 737-742
        • Lijmer J.G.
        • Leeflang M.
        • Bossuyt P.M.
        Proposals for a phased evaluation of medical tests.
        Med Decis Making. 2009; 29: E13-E21
        • Sackett D.L.
        • Haynes R.B.
        The architecture of diagnostic research.
        BMJ. 2002; 324: 539-541
        • Bossuyt P.M.
        • Lijmer J.G.
        • Mol B.W.
        Randomised comparisons of medical tests: sometimes invalid, not always efficient.
        Lancet. 2000; 356: 1844-1847
        • Thorpe K.E.
        • Zwarenstein M.
        • Oxman A.D.
        • Treweek S.
        • Furberg C.D.
        • Altman D.G.
        • et al.
        A pragmatic-explanatory continuum indicator summary (PRECIS): a tool to help trial designers.
        J Clin Epidemiol. 2009; 62: 464-475
        • Patsopoulos N.A.
        A pragmatic view on pragmatic trials.
        Dialogues Clin Neurosci. 2011; 13: 217-224
        • Ford I.
        • Norrie J.
        Pragmatic trials.
        N Engl J Med. 2016; 375: 454-463
        • Sox H.C.
        • Lewis R.J.
        Pragmatic trials: practical answers to "real world" questions.
        JAMA. 2016; 316: 1205-1206
        • Borgerson K.
        Are explanatory trials ethical? Shifting the burden of justification in clinical trial design.
        Theor Med Bioeth. 2013; 34: 293-308
        • Korevaar D.A.
        • van Enst W.A.
        • Spijker R.
        • Bossuyt P.M.
        • Hooft L.
        Reporting quality of diagnostic accuracy studies: a systematic review and meta-analysis of investigations on adherence to STARD.
        Evid Based Med. 2014; 19: 47-54
        • Bossuyt P.M.
        • Reitsma J.B.
        • Bruns D.E.
        • Gatsonis C.A.
        • Glasziou P.P.
        • Irwig L.
        • et al.
        Stard 2015: an updated list of essential items for reporting diagnostic accuracy studies.
        BMJ. 2015; 351: h5527