Journal of Clinical Epidemiology
Volume 59, Issue 10 , Pages 1040-1048, October 2006

A simple and valid tool distinguished efficacy from effectiveness studies

  • Gerald Gartlehner

      Affiliations

    • University of North Carolina at Chapel Hill, Cecil G. Sheps Center for Health Services Research, 725 Martin Luther King Jr. Boulevard, Chapel Hill, NC 27599, USA
    • Corresponding Author InformationCorresponding author. Tel. 919-966-7102; fax: 919-966-1634.
    • ,
  • Richard A. Hansen

      Affiliations

    • University of North Carolina, School of Pharmacy, Division of Pharmaceutical Policy & Evaluative Sciences, Chapel Hill, NC, USA
    • ,
  • Daniel Nissman

      Affiliations

    • Medical University of South Carolina – Department of Radiology, SC, USA
    • ,
  • Kathleen N. Lohr

      Affiliations

    • RTI International, Research Triangle Park, NC, USA
    • ,
  • Timothy S. Carey

      Affiliations

    • University of North Carolina, Cecil G. Sheps Center for Health Services Research, Chapel Hill, NC, USA

Accepted 22 January 2006. published online 10 August 2006.

Abstract 

Objective

To propose and test a simple instrument based on seven criteria of study design to distinguish effectiveness (pragmatic) from efficacy (explanatory) trials.

Study Design

Currently no validated definition of effectiveness studies exists. We asked the directors of 12 Evidence-based Practice Centers to select six studies each: four that they considered to be examples of effectiveness trials and two considered efficacy studies. We then applied our proposed criteria to test the construct validity using the selected studies as if they had been identified by a gold standard.

Results

Based on the rationale to identify effectiveness studies reliably with minimal false positives (i.e., a high specificity), a cutoff of six criteria produced the most desirable balance between sensitivity and specificity. This setting produced a specificity of 0.83 and a sensitivity of 0.72.

Conclusion

When applied in a standardized manner, our proposed criteria can provide a valid and simple tool to distinguish effectiveness from efficacy studies. The applicability of systematic reviews can improve when analysts place more emphasis on the generalizability of included studies. In addition, clinicians can also use our criteria to determine the external validity of individual studies, given an appropriate population of interest.

Keywords: Effectiveness, Efficacy, Pragmatic studies, Explanatory studies, Systematic reviews, Methods study

To access this article, please choose from the options below

Login to an existing account or Register a new account.

  • Purchase this article for 31.50 USD (You must login/register to purchase this article)

    Online access for 24 hours. The PDF version can be downloaded as your permanent record.

  • Subscribe to this title

    Get unlimited online access to this article and all other articles in this title 24/7 for one year.

  • Claim access now

    For current subscribers with Society Membership or Account Number.

  • Visit SciVerse ScienceDirect to see if you have access via your institution.
 

 Funding for this research was provided to the RTI-UNC EPC through a contract from the AHRQ to RTI International (contract number 290-02-0016). The funding source had no involvement in the design and conduct of the study, and in the analysis and interpretation of data.

PII: S0895-4356(06)00119-3

doi:10.1016/j.jclinepi.2006.01.011

Journal of Clinical Epidemiology
Volume 59, Issue 10 , Pages 1040-1048, October 2006

Article Tools