GRADE Series - Sharon Straus, Rachel Churchill and Sasha Shepperd, Guest Editors| Volume 64, ISSUE 12, P1283-1293, December 01, 2011

GRADE guidelines 6. Rating the quality of evidence—imprecision

      Abstract

      GRADE suggests that examination of 95% confidence intervals (CIs) provides the optimal primary approach to decisions regarding imprecision. For practice guidelines, rating down the quality of evidence (i.e., confidence in estimates of effect) is required if clinical action would differ if the upper versus the lower boundary of the CI represented the truth. An exception to this rule occurs when an effect is large, and consideration of CIs alone suggests a robust effect, but the total sample size is not large and the number of events is small. Under these circumstances, one should consider rating down for imprecision. To inform this decision, one can calculate the number of patients required for an adequately powered individual trial (termed the “optimal information size” [OIS]). For continuous variables, we suggest a similar process, initially considering the upper and lower limits of the CI, and subsequently calculating an OIS.
      Systematic reviews require a somewhat different approach. If the 95% CI excludes a relative risk (RR) of 1.0, and the total number of events or patients exceeds the OIS criterion, precision is adequate. If the 95% CI includes appreciable benefit or harm (we suggest an RR of under 0.75 or over 1.25 as a rough guide) rating down for imprecision may be appropriate even if OIS criteria are met.

      Keywords

      To read this article in full you will need to make a payment

      References

      1. A randomised, blinded, trial of clopidogrel versus aspirin in patients at risk of ischaemic events (CAPRIE). CAPRIE Steering Committee.
        Lancet. 1996; 348: 1329-1339
        • Poldermans D.
        • Boersma E.
        • Bax J.J.
        • et al.
        The effect of bisoprolol on perioperative mortality and myocardial infarction in high-risk patients undergoing vascular surgery. Dutch Echocardiographic Cardiac Risk Evaluation Applying Stress Echocardiography Study Group.
        N Engl J Med. 1999; 341: 1789-1794
        • Pocock S.J.
        • Hughes M.D.
        Practical problems in interim analyses, with particular regard to estimation.
        Control Clin Trials. 1989; 10: 209S-221S
        • Montori V.M.
        • Devereaux P.J.
        • Adhikari N.K.
        • et al.
        Randomized trials stopped early for benefit: a systematic review.
        JAMA. 2005; 294: 2203-2209
        • Bassler D.
        • Briel M.
        • Montori V.M.
        • et al.
        Stopping randomized trials early for benefit and estimation of treatment effects: systematic review and meta-regression analysis.
        JAMA. 2010; 303: 1180-1187
        • Teo K.K.
        • Yusuf S.
        • Collins R.
        • et al.
        Effects of intravenous magnesium in suspected acute myocardial infarction: overview of randomised trials.
        BMJ. 1991; 303: 1499-1503
      2. ISIS-4: a randomised factorial trial assessing early oral captopril, oral mononitrate, and intravenous magnesium sulphate in 58,050 patients with suspected acute myocardial infarction. ISIS-4 (Fourth International Study of Infarct Survival) Collaborative Group.
        Lancet. 1995; 345: 669-685
        • Abuissa H.
        • Jones P.G.
        • Marso S.P.
        • et al.
        Angiotensin-converting enzyme inhibitors or angiotensin receptor blockers for prevention of type 2 diabetes: a meta-analysis of randomized clinical trials.
        J Am Coll Cardiol. 2005; 46: 821-826
        • Bosch J.
        • Yusuf S.
        • Gerstein H.C.
        • et al.
        Effect of ramipril on the incidence of diabetes.
        N Engl J Med. 2006; 355: 1551-1562
        • Devereaux P.J.
        • Beattie W.S.
        • Choi P.T.
        • et al.
        How strong is the evidence for the use of perioperative beta blockers in non-cardiac surgery? Systematic review and meta-analysis of randomised controlled trials.
        BMJ. 2005; 331: 313-321
        • Bangalore S.
        • Wetterslev J.
        • Pranesh S.
        • et al.
        Perioperative beta blockers in patients having non-cardiac surgery: a meta-analysis.
        Lancet. 2008; 372: 1962-1976
        • Yusuf S.
        • Collins R.
        • MacMahon S.
        • et al.
        Effect of intravenous nitrates on mortality in acute myocardial infarction: an overview of the randomised trials.
        Lancet. 1988; 1: 1088-1092
      3. GISSI-3: effects of lisinopril and transdermal glyceryl trinitrate singly and together on 6-week mortality and ventricular function after acute myocardial infarction. Gruppo Italiano per lo Studio della Sopravvivenza nell'infarto Miocardico.
        Lancet. 1994; 343: 1115-1122
        • Imperiale T.F.
        • Petrulis A.S.
        A meta-analysis of low-dose aspirin for the prevention of pregnancy-induced hypertensive disease.
        JAMA. 1991; 266: 260-264
      4. CLASP: a randomised trial of low-dose aspirin for the prevention and treatment of pre-eclampsia among 9364 pregnant women. CLASP (Collaborative Low-dose Aspirin Study in Pregnancy) Collaborative Group.
        Lancet. 1994; 343: 619-629
      5. Human albumin administration in critically ill patients: systematic review of randomised controlled trials. Cochrane Injuries Group Albumin Reviewers.
        BMJ. 1998; 317: 235-240
        • Finfer S.
        • Bellomo R.
        • Boyce N.
        • et al.
        A comparison of albumin and saline for fluid resuscitation in the intensive care unit.
        N Engl J Med. 2004; 350: 2247-2256
        • Trikalinos T.A.
        • Churchill R.
        • Ferri M.
        • et al.
        Effect sizes in cumulative meta-analyses of mental health randomized trials evolved over time.
        J Clin Epidemiol. 2004; 57: 1124-1130
        • Pogue J.M.
        • Yusuf S.
        Cumulating evidence from randomized trials: utilizing sequential monitoring boundaries for cumulative meta-analysis.
        Control Clin Trials. 1997; 18 (discussion 661–666): 580-593
        • Alonso-Coello P.
        • Zhou Q.
        • Martinez-Zapata M.J.
        • et al.
        Meta-analysis of flavonoids for the treatment of haemorrhoids.
        Br J Surg. 2006; 93: 909-920
        • Gafter-Gvili A.
        • Fraser A.
        • Paul M.
        • et al.
        Meta-analysis: antibiotic prophylaxis reduces mortality in neutropenic patients.
        Ann Intern Med. 2005; 142: 979-995
        • Murad M.H.
        • Flynn D.N.
        • Elamin M.B.
        • et al.
        Endarterectomy vs stenting for carotid artery stenosis: a systematic review and meta-analysis.
        J Vasc Surg. 2008; 48: 487-493
        • Devereaux P.J.
        • Yang H.
        • Yusuf S.
        • et al.
        Effects of extended-release metoprolol succinate in patients undergoing non-cardiac surgery (POISE trial): a randomised controlled trial.
        Lancet. 2008; 371: 1839-1847
        • Butterworth A.D.
        • Thomas A.G.
        • Akobeng A.K.
        Probiotics for induction of remission in Crohn's disease.
        Cochrane Database of Systematic Reviews. 2008; (Issue 3. Art. No.: CD006634. doi:10.1002/14651858.CD006634.pub2)
        • Quon B.S.
        • Gan W.Q.
        • Sin D.D.
        Contemporary management of acute exacerbations of COPD: a systematic review and metaanalysis.
        Chest. 2008; 133: 756-766

      Linked Article