Advertisement

Assessing the impact of attrition in randomized controlled trials

      Abstract

      Objectives

      A survey of randomized controlled trials found that almost a quarter of trials had more than 10% of responses missing for the primary outcome. There are a number of ways in which data could be missing: the subject is unable to provide it, or they withdraw, or become lost to follow-up. Such attrition means that balance in baseline characteristics for those randomized may not be maintained in the subsample who has outcome data. For individual trials, if the attrition is systematic and linked to outcome, then this will result in biased estimates of the overall effect. It then follows that if such trials are combined in a meta-analysis, it will result in a biased estimate of the overall effect and be misleading. The aim of this study was to investigate the impact of attrition on baseline imbalance within individual trials and across multiple trials.

      Study Design and Setting

      In this article, we used individual patient data from a convenience sample of 10 trials evaluating interventions for the treatment of musculoskeletal disorders. Meta-analyses using the mean difference at baseline between the trial arms were carried out using individual patient data from these trials. The analyses were first carried out using all randomized participants and secondly only including participants with outcome data on the quality-of-life score. Meta-regression was carried out to evaluate whether the level of baseline imbalance was associated with the level of attrition.

      Results

      The overall attrition rates for the quality-of-life score ranged between 4% and 28% of the total randomized patients. All trials showed some level of differential attrition between the treatment arms, ranging from 1% to 14%. Attrition within the control group ranged from 3% to 25% and within the intervention group, it ranged from 0% to 31%. For individual trials, there was no indication that attrition altered the results in favor of either the treatment or the control. Forest plots highlighted that the attrition had some impact on the baseline imbalance for the primary outcome score as more heterogeneity was introduced (I-squared value of 0.4% for the initial data set vs. I-squared value of 16.9% for the analyzed data set). However, the standardized mean difference increased only slightly (from 0.01 to 0.03 with 95% confidence interval [CI]: −0.05, 0.10). Meta-regression showed little or no evidence of a significant dose–response relationship between the level of attrition and the baseline imbalance (coefficient 0.73, 95% CI: −0.81, 2.28).

      Conclusion

      Although, in theory, attrition can introduce selection bias in randomized trials, we did not find sufficient evidence to support this claim in our convenience sample of trials. However, the number of trials included was relatively small, which may have led to small but important differences in outcomes being missed. In addition, only 2 of 10 trials included had attrition levels greater than 15% suggesting a low level of potential bias. Meta-analyses and systematic reviews should always consider the impact of attrition on baseline imbalances and where possible any baseline imbalances in the analyzed data set and their impact on the outcomes reported.

      Keywords

      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'

      Subscribe:

      Subscribe to Journal of Clinical Epidemiology
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect

      References

        • Tierney J.F.
        • Stewart L.A.
        Investigating patient exclusion bias in meta-analysis.
        Int J Epidemiol. 2005; 34: 79
        • Dumville J.C.
        • Torgerson D.J.
        • Hewitt C.E.
        Reporting attrition in randomised controlled trials.
        BMJ. 2006; 332: 969
        • Hollis S.
        • Campbell F.
        What is meant by intention to treat analysis? Survey of published randomised controlled trials.
        BMJ. 1999; 319: 670
      1. Little RJA, Rubin DB. Statistical analysis with missing data. 1987.

        • Schulz K.F.
        • Grimes D.A.
        Sample size slippages in randomised trials: exclusions and the lost and wayward.
        Lancet. 2002; 359: 781-785
        • Leon A.C.
        • Mallinckrodt C.H.
        • Chuang-Stein C.
        • Archibald D.G.
        • Archer G.E.
        • Chartier K.
        Attrition in randomized controlled clinical trials: methodological issues in psychopharmacology.
        Biol Psychiatry. 2006; 59: 1001-1005
        • Valentine J.C.
        • McHugh C.M.
        The effects of attrition on baseline comparability in randomized experiments in education: a meta-analysis.
        Psychol Methods. 2007; 12: 268
        • Roberts C.
        • Torgerson D.J.
        Understanding controlled trials: baseline imbalance in randomised controlled trials.
        BMJ. 1999; 319: 185
        • Moher D.
        • Schulz K.F.
        • Altman D.G.
        The CONSORT statement: revised recommendations for improving the quality of reports of parallel group randomized trials.
        BMC Med Res Methodol. 2001; 1: 2
        • Trowman R.
        • Dumville J.C.
        • Torgerson D.J.
        • Cranny G.
        The impact of trial baseline imbalances should be considered in systematic reviews: a methodological case study.
        J Clin Epidemiol. 2007; 60: 1229-1233
        • Altman D.G.
        Comparability of randomized groups.
        Statistician. 1985; 34: 125-136
        • Preference Collaborative Review Group
        Patients' preferences within randomised trials: systematic review and patient level meta-analysis.
        BMJ. 2008; 337: a1864
        • Higgins J.
        • Green S.
        Cochrane handbook for systematic reviews of interventions.
        John Wiley & Sons Inc, 2008
        • Higgins J.P.T.
        • Thompson S.G.
        Quantifying heterogeneity in a meta-analysis.
        Stat Med. 2002; 21: 1539-1558
        • Higgins J.
        • Thompson S.G.
        • Deeks J.J.
        • Altman D.G.
        Measuring inconsistency in meta-analyses.
        BMJ. 2003; 327: 557
        • Moffett J.K.
        • Torgerson D.
        • Bell-Syer S.
        • Jackson D.
        • Llewlyn-Phillips H.
        • Farrin A.
        • et al.
        Randomised controlled trial of exercise for low back pain: clinical outcomes, costs, and preferences.
        BMJ. 1999; 319: 279
        • Donders A.R.T.
        • van der Heijden G.
        • Stijnen T.
        • Moons K.G.M.
        Review: a gentle introduction to imputation of missing values.
        J Clin Epidemiol. 2006; 59: 1087-1091
        • Carr J.L.
        • Klaber Moffett J.A.
        • Howarth E.
        • Richmond S.J.
        • Torgerson D.J.
        • Jackson D.A.
        • et al.
        A randomized trial comparing a group exercise programme for back pain patients with individual physiotherapy in a severely deprived area.
        Disabil Rehabil. 2005; 27: 929-937
        • Damask Direct Access to Magnetic Resonance Imaging Assessment for Suspect Knees Trial Team
        Effectiveness of GP access to magnetic resonance imaging of the knee: a randomised trial.
        Br J Gen Pract. 2008; 58: e1-e9
        • Hay E.M.
        • Thomas E.
        • Paterson S.M.
        • Dziedzic K.
        • Croft P.R.
        A pragmatic randomised controlled trial of local corticosteroid injection and physiotherapy for the treatment of new episodes of unilateral shoulder pain in primary care.
        BMJ. 2003; 62: 394
        • Klaber-Moffett J.
        • Jackson D.A.
        • Richmond S.
        • Hahn S.
        • Coulton S.
        • Farrin A.
        • et al.
        Randomised trial of a brief physiotherapy intervention compared with usual physiotherapy for neck pain patients: outcomes and patients' preference.
        BMJ. 2005; 330: 75
        • Klaber-Moffett J.
        • Jackson D.A.
        • Gardiner E.D.
        • Torgerson D.J.
        • Coulton S.
        • Eaton S.
        • et al.
        Randomized trial of two physiotherapy interventions for primary care neck and back pain patients: “McKenzie” vs brief physiotherapy pain management.
        Rheumatology (Oxford). 2006; 45: 1514
        • Salter G.C.
        • Roman M.
        • Bland M.J.
        • MacPherson H.
        Acupuncture for chronic neck pain: a pilot for a randomised controlled trial.
        BMC Musculoskelet Disord. 2006; 7: 99
        • Thomas K.J.
        • MacPherson H.
        • Thorpe L.
        • Brazier J.
        • Fitter M.
        • Campbell M.J.
        • et al.
        Randomised controlled trial of a short course of traditional acupuncture compared with usual care for persistent non-specific low back pain.
        BMJ. 2006; 333: 623
        • UK BEAM Trial Team
        United Kingdom back pain exercise and manipulation (UK BEAM) randomised trial: effectiveness of physical treatments for back pain in primary care.
        BMJ. 2004; 329: 1377
        • Watson J.
        • Helliwell P.
        • Morton V.
        • Adebajo A.
        • Dickson J.
        • Russell I.
        • et al.
        Shoulder acute pain in primary healthcare: is retraining effective for GP principals? SAPPHIRE—a randomized controlled trial.
        Rheumatology (Oxford). 2008; 47: 1795