Original article| Volume 136, P20-25, August 2021

Ok

# A method for calculating the fragility index of continuous outcomes

Published:March 04, 2021

## Highlights

• Fragility Index provides useful insight into how robust a statistically significant result really is.
• Classic computations of Fragility Index are limited to dichotomous variables.
• An iterative algorithm can be used to calculate Continuous Fragility Index (CFI) for continuous variables.
• In settings where original data is not available, a multiple simulation technique can be used to estimate CFI.
• The fragility of outcomes within the same study may vary considerably.

## Abstract

### Objective

Clinicians’ overdependence on p-values to determine significance in clinical trials is common yet potentially misleading. The Fragility Index (FI) describes how robust a significant result is by determining the number of events the statistical significance hinges on. However, this concept cannot be applied to nondichotomous variables. We describe a method to calculate a Continuous Fragility Index (CFI) for continuous variables. We further provide a method to estimate CFI when original data is not available.

### Study Design and Setting

An iterative substitution algorithm is described to calculate CFI prospectively from data or retrospectively from summary statistics and its response to variations in the data is reported. We then apply this method to a previously published review as a proof-of-concept.

### Results

The CFI increases linearly with sample size, logarithmically with mean difference, and decreases exponentially with standard deviation. Forty-eight studies were included of which 30 had significant non-dichotomous outcomes. CFI and FI were uncorrelated and mean CFI was significantly higher than FI (9 vs. 2, P< 0.001).

### Conclusion

Our algorithm extends fragility to continuous outcomes, expanding the applications of the fragility concept. The fragility of outcomes within a single study may vary based on variable type and should be evaluated independently.

## References

1. It's time to talk about ditching statistical significance.
Nature. 2019; 567: 283https://doi.org/10.1038/d41586-019-00874-8
• Walsh M
• Srinathan S
• McAuley D
The statistical significance of randomized controlled trial results is frequently fragile: a case for a fragility index.
J Clin Epidemiol. 2014; 67: 622-628
• Ruxton GD.
The unequal variance t-test is an underused alternative to Student's t-test and the Mann–Whitney U test.
Behav Ecol. 2006; 17: 688-690https://doi.org/10.1093/beheco/ark016
• Khan M
• Evaniew N
• Gichuru M
• et al.
The Fragility of Statistically Significant Findings From Randomized Trials in Sports Surgery: A Systematic Survey.
Am J Sports Med. 2017; 45: 2164-2170https://doi.org/10.1177/0363546516674469
• Team RC.
R: A language and environment for statistical computing.
2019
• Windish DM
• Huot SJ
• Green ML.
Medicine residents’ understanding of the biostatistics and results in the medical literature.
JAMA. 2007; 298: 1010-1022https://doi.org/10.1001/jama.298.9.1010
• Horton NJ
• Switzer SS.
Statistical Methods in the Journal.
N Engl J Med. 2005; 353: 1977-1979https://doi.org/10.1056/NEJM200511033531823
• Wasserstein RL
• Lazar NA.
The ASA's Statement on p-Values: Context, Process, and Purpose.
Am Stat. 2016; 70: 129-133https://doi.org/10.1080/00031305.2016.1154108
• Nuzzo R.
Scientific method: statistical errors.
Nature. 2014; 506: 150-152https://doi.org/10.1038/506150a
• Woods KL
• Fletcher S
• Roffe C
• Haider Y.
Intravenous magnesium sulphate in suspected acute myocardial infarction: results of the second Leicester Intravenous Magnesium Intervention Trial (LIMIT-2).
Lancet. 1992; 339: 1553-1558https://doi.org/10.1016/0140-6736(92)91828-V
• Ranstam J.
Why the P-value culture is bad and confidence intervals a better alternative.
Osteoarthr Cartil. 2012; 20: 805-808https://doi.org/10.1016/j.joca.2012.04.001
• Barnett AG
• Wren JD.
Examination of CIs in health and medical journals from 1976 to 2019: An observational study.
BMJ Open. 2019; 9e032506https://doi.org/10.1136/bmjopen-2019-032506
• Chavalarias D
• Wallach JD
• Li AHT
• Ioannidis JPA.
Evolution of reporting P values in the biomedical literature, 1990-2015.
JAMA - J Am Med Assoc. 2016; 315: 1141-1148https://doi.org/10.1001/jama.2016.1952
• Feinstein AR.
The unit fragility index: An additional appraisal of “statistical significance” for a contrast of two proportions.
J Clin Epidemiol. 1990; 43: 201-209https://doi.org/10.1016/0895-4356(90)90186-S
• Robinson CM
• Jenkins PJ
• White TO
• Ker A
• Will E.
Primary arthroscopic stabilization for a first-time anterior dislocation of the shoulder: A randomized, double-blind trial.
J Bone Jt Surg - Ser A. 2008; 90: 708-721https://doi.org/10.2106/JBJS.G.00679
• Zhao J
• Huangfu X
• He Y.
The role of medial retinaculum plication versus medial patellofemoral ligament reconstruction in combined procedures for recurrent patellar instability in adults.
Am J Sports Med. 2012; 40: 1355-1364https://doi.org/10.1177/0363546512439193
• Desnoyers A
• Wilson BE
• Amir E.
A critique of the fragility index.
Lancet Oncol. 2019; 20: e552https://doi.org/10.1016/S1470-2045(19)30583-2
• Carter RE
• McKie PM
• Storlie CB.
The Fragility Index: a P-value in sheep's clothing?.
Eur Heart J. 2016; 38: 346-348https://doi.org/10.1093/eurheartj/ehw495
• Acuna SA
• Sue-Chue-Lam C
• Dossa F.
The fragility index - p values reimagined, flaws and all.
JAMA Surg. 2019; 154: 674https://doi.org/10.1001/jamasurg.2019.0567