To the Editor:
In a recent article on the considerable variation of the number needed to treat (NNT), Wisløff et al. [
[1]
] used Monte Carlo simulations to compare the distribution of NNT with that of the relative risk (RR). The methods used have the following limitations and shortcomings.At first, to reach the trivial conclusion that observed values of RR have a regular continuous distribution, whereas the distribution of NNT is “irregular,” no Monte Carlo simulations are required. This conclusion follows directly from the known fact that the asymptotic distribution of RR and NNT is given by the lognormal and the inverse normal distribution, respectively [
[2]
]. The lognormal distribution has one mode, and the inverse normal distribution is bimodal [[3]
]. The properties of NNT have repeatedly been investigated and described before demonstrating undesirable statistical properties [2
, 4
, 5
, 6
]. Therefore, NNT should not be used as primary effect measure for data analysis [7
, 8
].Secondly, further explorations regarding the probability that the observed treatment effect is zero or negative when the true value is positive are discussed in detail for NNTs only. However, all situations that lead to a large probability that the observed treatment effect is zero or negative when the true value is positive in terms of NNTs lead to the same large probability for any other effect measure including RR. Thus, it is not a disadvantage of the NNT.
Thirdly, for the comparison of the variation of NNT and RR, it seems that histograms simply have been visually inspected. Only data situations are considered in which the distribution of RR approximately is symmetrical. However, RR has a skew distribution whereby the skewness depends on the standard error of log(RR). On the other hand, the distribution of the risk difference (RD) is always approximately symmetrical. By using the same simple visual inspection of histograms in data situations in which RR has a clear skew distribution, the conclusion that RR varies more than RD could be made. Thus, the simple visual inspection of a few histograms is insufficient. An appropriate quantitative analysis is required to compare the variation of effect measures.
A useful information given in the article by Wisløff et al. [
[1]
] is the conclusion that the true baseline risk is important to correctly interpret any measure of effectiveness, including RD and NNT. The explanation and discussion of this issue deserve more room, as this issue is not sufficiently explored in the existing NNT literature.In summary, previous research findings have been neglected leading to superfluous simulations. The general problem of insufficient power is described as negative feature of NNTs, although it affects every effect measure, and the method used to compare the variation of NNT and RR is insufficient. Nevertheless, I agree with the conclusions that NNTs should be used with caution and that the baseline risk is important for the correct interpretation of NNTs.
References
- Considerable variation in NNT: a study based on Monte Carlo simulations.J Clin Epidemiol. 2011; 64: 444-450
- A closer look at the distribution of number needed to treat (NNT): a Bayesian approach.Biostatistics. 2003; 4: 365-370
- 2nd edition. Continuous Univariate Distributions. Vol 1. Wiley, New York, NY1994
- A note on the number needed to treat.Control Clin Trials. 1999; 20: 439-447
- Number needed to treat (NNT): estimation of a measure of clinical benefit.Stat Med. 2001; 20: 3947-3962
- Bias of estimates of the number needed to treat.Stat Med. 2005; 24: 1837-1848
- Number needed to treat (NNT).in: Armitage P. Colton T. 2nd edition. Encyclopedia of Biostatistics. Vol 6. Wiley, Chichester, UK2005: 3752-3761
- Common problems related to the use of number needed to treat.J Clin Epidemiol. 2010; 63: 820-825
Article info
Publication history
Published online: January 24, 2012
Identification
Copyright
© 2012 Elsevier Inc. Published by Elsevier Inc. All rights reserved.
ScienceDirect
Access this article on ScienceDirectLinked Article
- Regarding adequate methods for comparison of effect measuresJournal of Clinical EpidemiologyVol. 65Issue 4
- PreviewWe are pleased to observe that we agree with Prof Dr Bender that baseline risk should be considered when effect measurers are reported [1,2]. When it comes to his critique, we are clearly aware that one can look at the number needed to treat (NNT) from a mathematical point of view, and that a main reason for the difficulties with this measure is that it is defined by inverting a difference. This creates a fundamental instability in the NNT. Our aim, however, is to illustrate the practical implications of this for medical studies in a simple way, and we believe we do this rather clearly.
- Full-Text
- Preview
