Why methods matter in a meta-analysis: a reappraisal showed inconclusive injury preventive effect of Nordic hamstring exercise

Published:September 11, 2021DOI:https://doi.org/10.1016/j.jclinepi.2021.09.007

      Highlights

      • Meta-analyses claim 50% injury risk reduction using the Nordic hamstring exercise.
      • By using more appropriate methodology, previous results could not be replicated.
      • The evidence underpinning the protective effect of Nordic hamstring exercise so far remains inconclusive.
      • According to The Grading of Recommendations Assessment, Development and Evaluation, Nordic hamstring exercise can be only conditionally recommended.

      Abstract

      Objectives

      The Nordic hamstring exercise (NHE) has been strongly recommended to reduce hamstring injuries in previous meta-analyses (50% reduction in risk of injury). To underline the importance and impact of adopting appropriate methodology for evidence synthesis, we revisited the study selection, reanalyzed and updated the findings of the most recent meta-analysis.

      Study Design and Setting

      Only randomized control trials (RCT) using NHE as one of the prevention arms were selected. Summary effects for risk ratios (RR) for original studies included in the earlier meta-analysis, and new studies identified (update), were re-estimated under the random-effects model and presented with 95% confidence intervals (CI) and prediction intervals (PI). Tentative recommendations were provided according to the Grading of Recommendations Assessment, Development and Evaluation.

      Results

      Only five RCTs out of the 15 studies included in the earlier meta-analysis randomized to NHE. Our update revealed one additional RCT. The point estimate (RR) for the five RCTs previously considered RCTs was 0.56 (95% CI, 0.20–1.52; 95% PI, 0.06–5.14, parametric, and 0.13–1.80, nonparametric). After the update, the RR was 0.59 (95% CI, 0.27–1.29; 95% PI 0.10–3.29, parametric, and 0.17–1.52, nonparametric).

      Conclusion

      Contrary to the conclusions of a recent meta-analysis, as well as earlier meta-analyses, by using more appropriate methodology, the evidence underpinning the protective effect of NHE so far remains inconclusive and mostly derived from high risk of bias RCTs. At best, only conditional recommendation can be provided (for soccer) and future RCTs are warranted.

      Keywords

      To read this article in full you will need to make a payment

      References

        • Vriend I
        • Gouttebarge V
        • Finch CF
        • van Mechelen W
        • Verhagen E.
        Intervention strategies used in sport injury prevention studies: a systematic review identifying studies applying the Haddon Matrix.
        Sports Med. 2017; 47: 2027-2043
        • McCall A
        • Pruna R
        • Van der Horst N
        • Dupont G
        • Buchheit M
        • Coutts AJ
        • et al.
        Exercise-based strategies to prevent muscle injury in male elite footballers: an expert-led Delphi Survey of 21 practitioners belonging to 18 teams from the big-5 European leagues.
        Sports Med. 2020; 50: 1667-1681
        • Al Attar WSA
        • Komir R
        • Alzubeadi A
        • Bukhari I
        • Ghulam H
        Limited implementation of the Nordic hamstring exercise in professional and semi-professional soccer.
        J Orthop Trauma Rehabil. 2021; 28: 1-7
        • O'Connor S
        • Lacey P.
        Can we improve coaches' injury prevention views and implementation practices in the community female Gaelic sport of camogie?.
        BMJ Open Sport Exerc Med. 2020; 6e000732
        • van der Horst N
        • van de Hoef S
        • van Otterloo P
        • Klein M
        • Brink M
        • Backx F
        Effective but not adhered to: how can we improve adherence to evidence-based hamstring injury prevention in amateur football?.
        Clin J Sport Med. 2021; 31: 42-48
        • Chesterton P
        • Tears C.
        The uptake of the Nordic hamstring exercise programme as an injury prevention strategy in professional cricket in the United Kingdom and barriers to implementation.
        Phys Ther Sport. 2021; 50: 1-6
        • van Dyk N
        • Behan FP
        • Whiteley R.
        Including the Nordic hamstring exercise in injury prevention programmes halves the rate of hamstring injuries: a systematic review and meta-analysis of 8459 athletes.
        Br J Sports Med. 2019; 53: 1362-1370
        • Al Attar WSA
        • Soomro N
        • Sinclair PJ
        • Pappas E
        • Sanders RH
        Effect of injury prevention programs that include the Nordic hamstring exercise on hamstring injury rates in soccer players: a systematic review and meta-analysis.
        Sports Med. 2017; 47: 907-916
        • Raya-GonzÁlez J
        • Castillo D
        • Clemente FM
        Injury prevention of hamstring injuries through exercise interventions: an umbrella review.
        J Sports Med Phys Fitness. 2021; (Online ahead of print)https://doi.org/10.23736/S0022-4707.21.11670-6
        • Vatovec R
        • Kozinc Z
        • Sarabon N.
        Exercise interventions to prevent hamstring injuries in athletes: a systematic review and meta-analysis.
        Eur J Sport Sci. 2020; 20: 992-1004
        • Silvers-Granelli H
        • Mandelbaum B
        • Adeniji O
        • Insler S
        • Bizzini M
        • Pohlig R
        • et al.
        Efficacy of the FIFA 11+ Injury Prevention Program in the collegiate male soccer player.
        Am J Sports Med. 2015; 43: 2628-2637
        • Reeves BC
        • Deeks JJ
        • Higgins JPT
        • Higgins JPT
        • Thomas J
        • Chandler J
        • Cumpston M
        • Li T
        • Page MJ
        • Welch VA
        • et al.
        Chapter 24. Including non-randomized studies on intervention effects.
        Cochrane handbook for systematic reviews of interventions. 6. version, Cochrane2021 (updated February 2021)
        • IntHout J
        • Ioannidis JP
        • Rovers MM
        • Goeman JJ.
        Plea for routinely presenting prediction intervals in meta-analysis.
        BMJ Open. 2016; 6e010247
        • Riley RD
        • Higgins JP
        • Deeks JJ.
        Interpretation of random effects meta-analyses.
        BMJ. 2011; 342: d549
        • Graham PL
        • Moran JL.
        Robust meta-analytic conclusions mandate the provision of prediction intervals in meta-analysis summaries.
        J Clin Epidemiol. 2012; 65: 503-510
        • Kontopantelis E
        • Springate DA
        • Reeves D.
        A re-analysis of the Cochrane Library data: the dangers of unobserved heterogeneity in meta-analyses.
        PLoS One. 2013; 8: e69930
        • Rothman KJ.
        Epidemiology: an introduction.
        Oxford University Press, 2012
        • Rothman KJ
        • Greenland S
        • Lash TL.
        Modern epidemiology.
        Lippincott Williams & Wilkins, 2008
      1. Schünemann H, Brożek J, Guyatt G, Oxman A. GRADE handbook for grading quality of evidence and strength of recommendations. Updated October 2013. ed The GRADE Working Group (Available from guidelinedevelopment.org/handbook, 14 May 2021), 2013.

        • Andrews JC
        • Schunemann HJ
        • Oxman AD
        • Pottie K
        • Meerpohl JJ
        • Coello PA
        • et al.
        GRADE guidelines: 15. Going from evidence to recommendation-determinants of a recommendation's direction and strength.
        J Clin Epidemiol. 2013; 66: 726-735
        • Van Calster B
        • Wynants L
        • Riley RD
        • van Smeden M
        • Collins GS.
        Methodology over metrics: current scientific standards are a disservice to patients and society.
        J Clin Epidemiol. 2021; (Online ahead of print)https://doi.org/10.1016/j.jclinepi.2021.05.018
      2. Haynes B. Can it work? Does it work? Is it worth it? The testing of healthcare interventions is evolving 1999;319(7211):652-653.

        • Godwin M
        • Ruhland L
        • Casson I
        • MacDonald S
        • Delva D
        • Birtwhistle R
        • et al.
        Pragmatic controlled clinical trials in primary care: the struggle between external and internal validity.
        BMC Med Res Methodol. 2003; 3: 28
        • Page MJ
        • McKenzie JE
        • Bossuyt PM
        • Boutron I
        • Hoffmann TC
        • Mulrow CD
        • et al.
        The PRISMA 2020 statement: an updated guideline for reporting systematic reviews.
        J Clin Epidemiol. 2021; 18: 1-16
        • Sterne JAC
        • Savovic J
        • Page MJ
        • Elbers RG
        • Blencowe NS
        • Boutron I
        • et al.
        RoB 2: a revised tool for assessing risk of bias in randomised trials.
        BMJ. 2019; 366: l4898
        • Efthimiou O.
        Practical guide to the meta-analysis of rare events.
        Evid Based Ment Health. 2018; 21: 72-76
        • Hasebe Y
        • Akasaka K
        • Otsudo T
        • Tachibana Y
        • Hall T
        • Yamamoto M
        Effects of Nordic hamstring exercise on hamstring injuries in high school soccer players: a randomized controlled trial.
        Int J Sports Med. 2020; 41: 154-160
        • Langan D
        • Higgins JPT
        • Jackson D
        • Bowden J
        • Veroniki AA
        • Kontopantelis E
        • et al.
        A comparison of heterogeneity variance estimators in simulated random-effects meta-analyses.
        Res Synth Methods. 2019; 10: 83-98
        • IntHout J
        • Ioannidis JP
        • Borm GF.
        The Hartung-Knapp-Sidik-Jonkman method for random effects meta-analysis is straightforward and considerably outperforms the standard DerSimonian-Laird method.
        BMC Med Res Methodol. 2014; 14: 25
        • Partlett C
        • Riley RD.
        Random effects meta-analysis: Coverage performance of 95% confidence and prediction intervals following REML estimation.
        Stat Med. 2017; 36: 301-317
        • Wang CC
        • Lee WC.
        A simple method to estimate prediction intervals and predictive distributions: summarizing meta-analyses beyond means and confidence intervals.
        Res Synth Methods. 2019; 10: 255-266
        • Viechtbauer W.
        Confidence intervals for the amount of heterogeneity in meta-analysis.
        Stat Med. 2007; 26: 37-52
        • Sebelien C
        • Stiller CH
        • Maher S
        • Qu X.
        Effects of implementing Nordic hamstring exercises for semi-professional soccer players in Norway.
        Orthop Pract. 2014; 26: 90-97
        • van der Horst N
        • Smits DW
        • Petersen J
        • Goedhart EA
        • Backx FJ.
        The preventive effect of the nordic hamstring exercise on hamstring injuries in amateur soccer players: a randomized controlled trial.
        Am J Sports Med. 2015; 43: 1316-1323
        • Petersen J
        • Thorborg K
        • Nielsen MB
        • Budtz-Jorgensen E
        • Holmich P.
        Preventive effect of eccentric training on acute hamstring injuries in men's soccer: a cluster-randomized controlled trial.
        Am J Sports Med. 2011; 39: 2296-2303
        • Gabbe BJ
        • Branson R
        • Bennell KL.
        A pilot randomised controlled trial of eccentric exercise to prevent hamstring injuries in community-level Australian Football.
        J Sci Med Sport. 2006; 9: 103-109
        • Engebretsen AH
        • Myklebust G
        • Holme I
        • Engebretsen L
        • Bahr R.
        Prevention of injuries among male soccer players: a prospective, randomized intervention study targeting players with previous injuries or reduced function.
        Am J Sports Med. 2008; 36: 1052-1060
        • Brooks JH
        • Fuller CW
        • Kemp SP
        • Reddin DB.
        Incidence, risk, and prevention of hamstring muscle injuries in professional rugby union.
        Am J Sports Med. 2006; 34: 1297-1306
        • Arnason A
        • Andersen TE
        • Holme I
        • Engebretsen L
        • Bahr R.
        Prevention of hamstring strains in elite soccer: an intervention study.
        Scand J Med Sci Sports. 2008; 18: 40-48
        • Seagrave 3rd, RA
        • Perez L
        • McQueeney S
        • Toby EB
        • Key V
        • Nelson JD
        Preventive effects of eccentric training on acute hamstring muscle injury in professional baseball.
        Orthop J Sports Med. 2014; 22325967114535351
        • van der Horst N
        • Smits DW
        • Petersen J
        • Goedhart EA
        • Backx FJ.
        The preventive effect of the Nordic hamstring exercise on hamstring injuries in amateur soccer players: study protocol for a randomised controlled trial.
        Inj Prev. 2014; 20: e8
        • Elerian AE
        • El-Sayyad MM
        • Dorgham HMA.
        Effect of pre-training and post-training Nordic exercise on hamstring injury prevention, recurrence, and severity in soccer players.
        Ann Rehabil Med. 2019; 43: 465-473
        • Lee H
        • Herbert RD
        • Lamb SE
        • Moseley AM
        • McAuley JH.
        Investigating causal mechanisms in randomised controlled trials.
        Trials. 2019; 20: 524
        • Lee KJ
        • Thompson SG.
        Flexible parametric models for random-effects distributions.
        Stat Med. 2008; 27: 418-434
        • Deeks JJ
        • Higgins JPT
        • Altman DG.
        • Higgins JPT
        • Thomas J
        • Chandler J
        • Cumpston M
        • Li T
        • Page MJ
        • Welch VA
        Chapter 10: Analysing data and undertaking meta-analyses.
        Cochrane handbook for systematic reviews of interventions. 6. version, Cochrane2021 (updated February 2021)
        • Higgins J
        • Thompson S
        • Deeks J
        • Altman D.
        Statistical heterogeneity in systematic reviews of clinical trials: a critical appraisal of guidelines and practice.
        J Health Serv Res Policy. 2002; 7: 51-61
        • Riley RD
        • Gates S
        • Neilson J
        • Alfirevic Z.
        Statistical methods can be improved within Cochrane pregnancy and childbirth reviews.
        J Clin Epidemiol. 2011; 64: 608-618
        • Borenstein M
        • Hedges LV
        • Higgins JP
        • Rothstein HR.
        A basic introduction to fixed-effect and random-effects models for meta-analysis.
        Res Synth Methods. 2010; 1: 97-111
        • Furlan AD
        • Pennick V
        • Bombardier C
        • van Tulder M
        Editorial Board CBRG. 2009 updated method guidelines for systematic reviews in the Cochrane Back Review Group.
        Spine (Phila Pa 1976). 2009; 34: 1929-1941
        • Mack CD
        • Kent RW
        • Coughlin MJ
        • Shiue KY
        • Weiss LJ
        • Jastifer JR
        • et al.
        Incidence of Lower Extremity Injury in the National Football League: 2015 to 2018.
        Am J Sports Med. 2020; 48: 2287-2294
        • Whalan M
        • Lovell R
        • McCunn R
        • Sampson JA.
        The incidence and burden of time loss injury in Australian men's sub-elite football (soccer): A single season prospective cohort study.
        J Sci Med Sport. 2019; 22: 42-47
        • Rees H
        • McCarthy Persson U
        • Delahunt E
        • Boreham C
        • Blake C.
        The burden of injury in field hockey: a secondary analysis of prospective cohort data.
        Scand J Med Sci Sports. 2021; 31: 884-893
        • Roe M
        • Murphy JC
        • Gissane C
        • Blake C.
        Time to get our four priorities right: an 8-year prospective investigation of 1326 player-seasons to identify the frequency, nature, and burden of time-loss injuries in elite Gaelic football.
        PeerJ. 2018; 6: e4895
        • Owoeye OBA
        • VanderWey MJ
        • Pike I.
        Reducing injuries in soccer (football): an umbrella review of best evidence across the epidemiological framework for prevention.
        Sports Med Open. 2020; 6: 46