Abstract
Objective
To provide GRADE guidance for assessing risk of bias across an entire body of evidence
consequent on missing data for systematic reviews of both binary and continuous outcomes.
Study Design and Setting
Systematic survey of published methodological research, iterative discussions, testing
in systematic reviews, and feedback from the GRADE Working Group.
Results
Approaches begin with a primary meta-analysis using a complete case analysis followed
by sensitivity meta-analyses imputing, in each study, data for those with missing
data, and then pooling across studies. For binary outcomes, we suggest use of “plausible
worst case” in which review authors assume that those with missing data in treatment
arms have proportionally higher event rates than those followed successfully. For
continuous outcomes, imputed mean values come from other studies within the systematic
review and the standard deviation (SD) from the median SDs of the control arms of
all studies.
Conclusions
If the results of the primary meta-analysis are robust to the most extreme assumptions
viewed as plausible, one does not rate down certainty in the evidence for risk of
bias due to missing participant outcome data. If the results prove not robust to plausible
assumptions, one would rate down certainty in the evidence for risk of bias.
Keywords
To read this article in full you will need to make a payment
Purchase one-time access:
Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online accessOne-time access price info
- For academic or personal research use, select 'Academic and Personal'
- For corporate R&D use, select 'Corporate R&D Professionals'
Subscribe:
Subscribe to Journal of Clinical EpidemiologyAlready a print subscriber? Claim online access
Already an online subscriber? Sign in
Register: Create an account
Institutional Access: Sign in to ScienceDirect
References
- GRADE guidelines: 4. Rating the quality of evidence–study limitations (risk of bias).J Clin Epidemiol. 2011; 64: 407-415
- GRADE guidelines: 3. Rating the quality of evidence.J Clin Epidemiol. 2011; 64: 401-406
- Higgins J.P.T. Green S S. Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0. The Cochrane Collaboration. 2011 (Available from www.handbook.cochrane.org)
- Addressing dichotomous data for participants excluded from trial analysis: a guide for systematic reviewers.PLoS One. 2013; 8: e57132
- Addressing continuous data for participants excluded from trial analysis: a guide for systematic reviewers.J Clin Epidemiol. 2013; 66: 1014-1021.e1
- Addressing continuous data measured with different instruments for participants excluded from trial analysis: a guide for systematic reviewers.J Clin Epidemiol. 2014; 67: 560-570
- Handling trial participants with missing outcome data when conducting a meta-analysis: a systematic survey of proposed approaches.Syst Rev. 2015; 4: 98
- Reporting, handling and assessing the risk of bias associated with missing participant data in systematic reviews: a methodological survey.BMJ Open. 2015; 5: e009368
- Reporting missing participant data in randomised trials: systematic survey of the methodological literature and a proposed guide.BMJ Open. 2015; 5: e008431
- Three challenges described for identifying participants with missing data in trials reports, and potential solutions suggested to systematic reviewers.J Clin Epidemiol. 2016; 76: 147-154
- Modified intention to treat reporting in randomised controlled trials: systematic review.BMJ. 2010; 340: c2697
- Deviation from intention to treat analysis in randomised trials and treatment effect estimates: meta-epidemiological study.BMJ. 2015; 350: h2445
- Assessing allocation concealment and blinding in randomised controlled trials: why bother?.Evid Based Nurs. 2001; 4: 4-6
- Inconsistent definitions for intention-to-treat in relation to missing outcome data: systematic review of the methods literature.PLoS One. 2012; 7: e49163
- Imputation methods for missing outcome data in meta-analysis of clinical trials.Clin Trials. 2008; 5: 225-239
- Allowing for uncertainty due to missing continuous outcome data in pairwise and network meta-analysis.Stat Med. 2015; 34: 721-741
- Potential impact on estimated treatment effects of information lost to follow-up in randomised controlled trials (LOST-IT): systematic review.BMJ. 2012; 344: e2809
- Allowing for uncertainty due to missing data in meta-analysis–part 1: two-stage methods.Stat Med. 2008; 27: 711-727
- Allowing for uncertainty due to missing data in meta-analysis–part 2: hierarchical models.Stat Med. 2008; 27: 728-745
- A Bayesian framework to account for uncertainty due to missing binary outcome data in pairwise meta-analysis.Stat Med. 2015; 34: 2062-2080
- Sampling-based approach to determining outcomes of patients lost to follow-up in antiretroviral therapy scale-up programs in Africa.JAMA. 2008; 300: 506-507
- Anticoagulation for the long-term treatment of venous thromboembolism in patients with cancer.Cochrane Database Syst Rev. 2014; 7: CD006650
- Low molecular weight heparin versus unfractionated heparin for perioperative thromboprophylaxis in patients with cancer.Cochrane Database Syst Rev. 2014; 6: CD009447
- Oral anticoagulation in patients with cancer who have no therapeutic or prophylactic indication for anticoagulation.Cochrane Database Syst Rev. 2014; 7: CD006466
- Parenteral anticoagulation in ambulatory patients with cancer.Cochrane Database Syst Rev. 2014; 12: CD006652
- Anticoagulation for people with cancer and central venous catheters.Cochrane Database Syst Rev. 2014; 10: CD006468
- Probiotics and synbiotics for the prevention of postoperative infections following abdominal surgery: a systematic review and meta-analysis of randomized controlled trials.J Hosp Infect. 2016; 92: 130-139
- Probiotics for the prevention of Clostridium difficile-associated diarrhea: a systematic review and meta-analysis.Ann Intern Med. 2012; 157: 878-888
- Culprit vessel versus immediate complete revascularization in patients with ST-segment myocardial infarction-a systematic review.Am Heart J. 2015; 170: 1133-1139
- Systematic review of percutaneous closure versus medical therapy in patients with cryptogenic stroke and patent foramen ovale.BMJ Open. 2014; 4: e004282
- Longer- versus shorter-duration dual-antiplatelet therapy after drug-eluting stent placement: a systematic review and meta-analysis.Ann Intern Med. 2015; 163: 118-126
- How to read clinical journals: V: to distinguish useful from useless or even harmful therapy.Can Med Assoc J. 1981; 124: 1156-1162
- Improving the interpretation of quality of life evidence in meta-analyses: the application of minimal important difference units.Health Qual Life Outcomes. 2010; 8: 116
- GRADE guidelines: 13. Preparing summary of findings tables and evidence profiles-continuous outcomes.J Clin Epidemiol. 2013; 66: 173-183
- Measuring health-related quality of life.Ann Intern Med. 1993; 118: 622-629
- Pooling health-related quality of life outcomes in meta-analysis-a tutorial and review of methods for enhancing interpretability.Res Synth Methods. 2011; 2: 188-203
- Impact of missing participant data for dichotomous outcomes on pooled effect estimates in systematic reviews: a protocol for a methodological study.Syst Rev. 2014; 3: 137
Article info
Publication history
Published online: May 18, 2017
Accepted:
May 2,
2017
Footnotes
Conflict of interest: All the authors have completed the ICMJE uniform disclosure form and declare no support from any organization for the submitted work and no financial relationships with any organizations that might have an interest in the submitted work in the previous 3 years. They declare being involved in previous publications making recommendations on the topic missing participant outcome data.
Funding: This study is part of a project on addressing missing trial participant data in systematic reviews funded by the Cochrane Collaboration. P.A.-C. was funded by a Miguel Servet research contract from the Instituto de Salud Carlos III (CP16/00137). A.G.M. was funded by a Fellowship in Guidelines Methodology by European Respiratory Society (MTF 2015-01). The funders were not involved in study design and the collection, analysis, and interpretation of data and the writing of the article and the decision to submit it for publication. The researchers are independent from funders and had full access to all the data.
Identification
Copyright
© 2017 Elsevier Inc. All rights reserved.
