Highlights
- •Fragility Index provides useful insight into how robust a statistically significant result really is.
- •Classic computations of Fragility Index are limited to dichotomous variables.
- •An iterative algorithm can be used to calculate Continuous Fragility Index (CFI) for continuous variables.
- •In settings where original data is not available, a multiple simulation technique can be used to estimate CFI.
- •The fragility of outcomes within the same study may vary considerably.
Abstract
Objective
Clinicians’ overdependence on p-values to determine significance in clinical trials
is common yet potentially misleading. The Fragility Index (FI) describes how robust
a significant result is by determining the number of events the statistical significance
hinges on. However, this concept cannot be applied to nondichotomous variables. We
describe a method to calculate a Continuous Fragility Index (CFI) for continuous variables.
We further provide a method to estimate CFI when original data is not available.
Study Design and Setting
An iterative substitution algorithm is described to calculate CFI prospectively from
data or retrospectively from summary statistics and its response to variations in
the data is reported. We then apply this method to a previously published review as
a proof-of-concept.
Results
The CFI increases linearly with sample size, logarithmically with mean difference,
and decreases exponentially with standard deviation. Forty-eight studies were included
of which 30 had significant non-dichotomous outcomes. CFI and FI were uncorrelated
and mean CFI was significantly higher than FI (9 vs. 2, P< 0.001).
Conclusion
Our algorithm extends fragility to continuous outcomes, expanding the applications
of the fragility concept. The fragility of outcomes within a single study may vary
based on variable type and should be evaluated independently.
Keywords
To read this article in full you will need to make a payment
Purchase one-time access:
Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online accessOne-time access price info
- For academic or personal research use, select 'Academic and Personal'
- For corporate R&D use, select 'Corporate R&D Professionals'
Subscribe:
Subscribe to Journal of Clinical EpidemiologyAlready a print subscriber? Claim online access
Already an online subscriber? Sign in
Register: Create an account
Institutional Access: Sign in to ScienceDirect
References
- It's time to talk about ditching statistical significance.Nature. 2019; 567: 283https://doi.org/10.1038/d41586-019-00874-8
- The statistical significance of randomized controlled trial results is frequently fragile: a case for a fragility index.J Clin Epidemiol. 2014; 67: 622-628
- The unequal variance t-test is an underused alternative to Student's t-test and the Mann–Whitney U test.Behav Ecol. 2006; 17: 688-690https://doi.org/10.1093/beheco/ark016
- The Fragility of Statistically Significant Findings From Randomized Trials in Sports Surgery: A Systematic Survey.Am J Sports Med. 2017; 45: 2164-2170https://doi.org/10.1177/0363546516674469
- R: A language and environment for statistical computing.2019
- Medicine residents’ understanding of the biostatistics and results in the medical literature.JAMA. 2007; 298: 1010-1022https://doi.org/10.1001/jama.298.9.1010
- Statistical Methods in the Journal.N Engl J Med. 2005; 353: 1977-1979https://doi.org/10.1056/NEJM200511033531823
- The ASA's Statement on p-Values: Context, Process, and Purpose.Am Stat. 2016; 70: 129-133https://doi.org/10.1080/00031305.2016.1154108
- Scientific method: statistical errors.Nature. 2014; 506: 150-152https://doi.org/10.1038/506150a
- Intravenous magnesium sulphate in suspected acute myocardial infarction: results of the second Leicester Intravenous Magnesium Intervention Trial (LIMIT-2).Lancet. 1992; 339: 1553-1558https://doi.org/10.1016/0140-6736(92)91828-V
- Why the P-value culture is bad and confidence intervals a better alternative.Osteoarthr Cartil. 2012; 20: 805-808https://doi.org/10.1016/j.joca.2012.04.001
- Examination of CIs in health and medical journals from 1976 to 2019: An observational study.BMJ Open. 2019; 9e032506https://doi.org/10.1136/bmjopen-2019-032506
- Evolution of reporting P values in the biomedical literature, 1990-2015.JAMA - J Am Med Assoc. 2016; 315: 1141-1148https://doi.org/10.1001/jama.2016.1952
- The unit fragility index: An additional appraisal of “statistical significance” for a contrast of two proportions.J Clin Epidemiol. 1990; 43: 201-209https://doi.org/10.1016/0895-4356(90)90186-S
- Primary arthroscopic stabilization for a first-time anterior dislocation of the shoulder: A randomized, double-blind trial.J Bone Jt Surg - Ser A. 2008; 90: 708-721https://doi.org/10.2106/JBJS.G.00679
- The role of medial retinaculum plication versus medial patellofemoral ligament reconstruction in combined procedures for recurrent patellar instability in adults.Am J Sports Med. 2012; 40: 1355-1364https://doi.org/10.1177/0363546512439193
- A critique of the fragility index.Lancet Oncol. 2019; 20: e552https://doi.org/10.1016/S1470-2045(19)30583-2
- The Fragility Index: a P-value in sheep's clothing?.Eur Heart J. 2016; 38: 346-348https://doi.org/10.1093/eurheartj/ehw495
- The fragility index - p values reimagined, flaws and all.JAMA Surg. 2019; 154: 674https://doi.org/10.1001/jamasurg.2019.0567
- A critique of the fragility index.Lancet Oncol. 2019; 20: e553https://doi.org/10.1016/S1470-2045(19)30581-9
- A critique of the fragility index – Authors’ reply.Lancet Oncol. 2019; 20: e554https://doi.org/10.1016/S1470-2045(19)30580-7
Article info
Publication history
Published online: March 04, 2021
Accepted:
February 24,
2021
Received in revised form:
February 9,
2021
Received:
August 21,
2020
Footnotes
Source of Funding: There was no significant funding for this work.
Declarations of interest: None
Identification
Copyright
© 2021 Elsevier Inc. All rights reserved.
