Abstract
Objective
To describe a systematic approach for identifying, reporting, and synthesizing information
to allow consistent and transparent consideration of the applicability of the evidence
in a systematic review according to the Population, Intervention, Comparator, Outcome,
Setting domains.
Study Design and Setting
Comparative effectiveness reviews need to consider whether available evidence is applicable
to specific clinical or policy questions to be useful to decision makers. Authors
reviewed the literature and developed guidance for the Effective Health Care program.
Results
Because applicability depends on the specific questions and needs of the users, it
is difficult to devise a valid uniform scale for rating the overall applicability
of individual studies or body of evidence. We recommend consulting stakeholders to
identify the factors most relevant to applicability for their decisions. Applicability
should be considered separately for benefits and harms. Observational studies can
help determine whether trial populations and interventions are representative of “real
world” practice. Reviewers should describe differences between available evidence
and the ideally applicable evidence for the question being asked and offer a qualitative
judgment about the importance and potential effect of those differences.
Conclusion
Careful consideration of applicability may improve the usefulness of systematic reviews
in informing practice and policy.
Keywords
To read this article in full you will need to make a payment
Purchase one-time access:
Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online accessOne-time access price info
- For academic or personal research use, select 'Academic and Personal'
- For corporate R&D use, select 'Corporate R&D Professionals'
Subscribe:
Subscribe to Journal of Clinical EpidemiologyAlready a print subscriber? Claim online access
Already an online subscriber? Sign in
Register: Create an account
Institutional Access: Sign in to ScienceDirect
References
- Report to the president and the congress on comparative effectiveness research.(Available at) (Accessed June 30, 2009)
- Higgins J.P.T. Green S. Cochrane handbook for systematic reviews of interventions version 5.0.2. The Cochrane Collaboration, 2009 (Available at) (Accessed September 2009)
- Experimental and quasi-experimental designs for generalized causal inference.Houghton Mifflin, Boston, MA2002
- What is “quality of evidence” and why is it important to clinicians?.BMJ. 2008; 336: 995-998
- External validity of randomised controlled trials: “to whom do the results of this trial apply?”.Lancet. 2005; 365: 82-93
- AHRQ series paper 4: assessing harms when comparing medical interventions: AHRQ and the effective health-care program.J Clin Epidemiol. 2010; 63: 502-512
- Checklist for the qualitative evaluation of clinical studies with particular focus on external validity and model validity.BMC Med Res Methodol. 2006; 6: 56
- Evaluating the relevance, generalization, and applicability of research: issues in external validation and translation methodology.Eval Health Prof. 2006; 29: 126-153
- Applicability and generalisability of published results of randomised controlled trials and non-randomised studies evaluating four orthopaedic procedures: methodological systematic review.BMJ. 2009; 339: b4538
- AHRQ series paper 5: grading the strength of a body of evidence when comparing medical interventions—agency for healthcare research and quality and the effective health-care program.J Clin Epidemiol. 2010; 63: 513-523
- AHRQ series commentary 1: rating the evidence in comparative effectiveness reviews.J Clin Epidemiol. 2010; 63: 474-475
- Current processes of the U.S. Preventive Services Task Force: refining evidence-based recommendation development.Ann Intern Med. 2007; 147: 117-122
- Effect of alendronate on risk of fracture in women with low bone density but without vertebral fractures: results from the Fracture Intervention Trial.JAMA. 1998; 280: 2077-2082
- Variations between clinical trial participants and Medicare beneficiaries in evidence used for Medicare national coverage decisions.Arch Intern Med. 2008; 168: 136-140
- Comparative effectiveness of percutaneous coronary interventions and coronary artery bypass grafting for coronary artery disease. Comparative Effectiveness Review No. 9. (Prepared by Stanford-UCSF Evidence-based Practice Center under Contract No. 290-02-0017.).Agency for Healthcare Research and Quality, Rockville, MD2007 (Available at) (Accessed February 23, 2011)
- Effects of conjugated equine estrogen in postmenopausal women with hysterectomy: the Women’s Health Initiative randomized controlled trial.JAMA. 2004; 291: 1701-1712
- Comparative effectiveness of second-generation antidepressants in the pharmacologic treatment of adult depression. Comparative Effectiveness Review No. 7. (Prepared by RTI International-University of North Carolina Evidence-based Practice Center under Contract No. 290-02-0016.).Agency for Healthcare Research and Quality, Rockville, MD2007 (Available at) (Accessed February 23, 2011)
- Effectiveness of weight management programs in children and adolescents. Evidence Report/Technology Assessment No. 170 (Prepared by the Oregon Evidence-based Practice Center under Contract No. 290-02-0024). AHRQ Publication No. 08-E014.Agency for Healthcare Research and Quality, Rockville, MD2008
- Translating efficacy into effectiveness in antiretroviral therapy: beyond the pill count.Drugs. 2007; 67: 1969-1979
- Interactive effects of iron and zinc on biochemical and functional outcomes in supplementation trials.Am J Clin Nutr. 2005; 82: 5-12
- Variation in carotid endarterectomy mortality in the Medicare population.JAMA. 1998; 279: 1278-1281
- Duloxetine in the acute and long-term treatment of major depressive disorder: a placebo- and paroxetine-controlled trial.Eur Neuropsychopharmacol. 2004; 14: 457-470
- Intravenous magnesium for acute myocardial infarction.Cochrane Database Syst Rev. 2007; (Art. No.: CD002755)
- Problems with use of composite end points in cardiovascular trials: systematic review of randomised controlled trials.BMJ. 2007; 334: 786
- The impact of high-risk patients on the results of clinical trials.J Clin Epidemiol. 1997; 50: 1089-1098
- Drug class review of Alzheimer’s drugs.Final Report. 2006; (Available at) (Accessed February 23, 2011)
Humphrey L, Chan BKS, Detlefsen S, Helfand M, Screening for breast cancer. Prepared by Oregon Health Sciences University under Contract No. 290-97-0018. Rockville, MD. Agency for Healthcare Research and Quality. Available at: http://www.ncbi.nlm.nih.gov/books/bv.fcgi?rid=hstat3.chapter.27509. Accessed August 2002
- Comparison of endovascular and open surgical repairs for abdominal aortic aneurysm. Evidence Report/Technology Assessment No. 144. (Prepared by the University of Minnesota Evidence-based Practice Center under Contract No. 290-02-0009.) AHRQ Publication No. 06-E017.Agency for Healthcare Research and Quality, Rockville, MDAugust 2006
- Evidence-based medicine, heterogeneity of treatment effects, and the trouble with averages.Milbank Q. 2004; 82: 661-687
- Factors that can affect the external validity of randomised controlled trials.PLoS Clin Trials. 2006; 1: e9
- Lipid modification: cardiovascular risk assessment and the modification of blood lipids for the primary and secondary prevention of cardiovascular disease.NICE, London, UK2008 (Available at)
- Practical clinical trials: increasing the value of clinical research for decision making in clinical and health policy.JAMA. 2003; 290: 1624-1632
- Pragmatic controlled clinical trials in primary care: the struggle between external and internal validity.BMC Med Res Methodol. 2003; 3: 28
- Creating and synthesizing evidence with decision makers in mind: integrating evidence from clinical trials and other study designs.Med Care. 2007; 45: S16-S22
- A simple and valid tool distinguished efficacy from effectiveness studies.J Clin Epidemiol. 2006; 59: 1040-1048
- A pragmatic-explanatory continuum indicator summary (PRECIS): a tool to help trial designers.J Clin Epidemiol. 2009; 62: 464-475
- Meta-analysis of individual participant data: rationale, conduct, and reporting.BMJ. 2010; 340: c221
- Carotid endarterectomy for asymptomatic carotid stenosis.Cochrane Database Syst Rev. 2005; (Art. No.: CD001923)
- Evidence-based medicine—how to practice and teach EBM.Churchill Livingstone, New York, NY1997
- Comparative effectiveness of therapies for clinically localized prostate cancer. Comparative Effectiveness Review No. 13. (Prepared by Minnesota Evidence-based Practice Center under Contract No. 290-02-0009.).Agency for Healthcare Research and Quality, Rockville, MDFebruary 2008 (Available at) (Accessed February 23, 2011)
- Radical prostatectomy versus watchful waiting in early prostate cancer.N Engl J Med. 2005; 12: 1977-1984
- Radical prostatectomy versus expectant treatment for early carcinoma of the prostate. Twenty-three year followup of a prospective randomized study.Scan J Urol Nephrol Suppl. 1995; 172: 65-72
Article info
Publication history
Published online: April 04, 2011
Accepted:
November 5,
2010
Footnotes
Funding support from the Agency for Healthcare Research and Quality’s Effective Health Care Program.
Disclaimer: The findings and conclusions in this document are those of the authors, who are responsible for its contents; the findings and conclusions do not necessarily represent the views of Agency for Healthcare Research and Quality, the Veterans Health Administration, or the Health Services Research and Development Service. Therefore, no statement in this report should be construed as an official position of these entities, the US Department of Health and Human Services, or the US Department of Veterans Affairs.
Identification
Copyright
Published by Elsevier Inc.
