Abstract
Objective
To determine the statistical power to detect potentially clinically significant differences
in serious adverse events between drug therapies reported in a sample of randomized
controlled trials (RCTs).
Study Design and Setting
Systematic review of RCTs with positive efficacy endpoint and at least a twofold difference
in the proportion of patients with serious adverse events between treatment groups
from six major journals. The power of each study to detect statistically significant
differences in serious adverse events was calculated.
Results
Of the six included trials, all performed statistical analysis on adverse events without
disclosure of the statistical power for detecting the reported differences between
groups. The power of each study to detect the reported differences in adverse events
was calculated and yielded values ranging from 0.07 to 0.37 among trials with non–statistically
significant differences.
Conclusion
Statistical testing for differences in the proportion of patients experiencing an
adverse event is common in RCTs; non–statistically significant differences are associated
with low statistical power. A high probability of type II error may lead to erroneous
clinical inference resulting in harm. The statistical power for nonsignificant tests
should be considered in the interpretation of results.
Keywords
To read this article in full you will need to make a payment
Purchase one-time access:
Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online accessOne-time access price info
- For academic or personal research use, select 'Academic and Personal'
- For corporate R&D use, select 'Corporate R&D Professionals'
Subscribe:
Subscribe to Journal of Clinical EpidemiologyAlready a print subscriber? Claim online access
Already an online subscriber? Sign in
Register: Create an account
Institutional Access: Sign in to ScienceDirect
References
- The importance of beta, the type II error and sample size in the design and interpretation of the randomized control trial. Survey of 71 “negative” trials.N Engl J Med. 1978; 299: 690-694
- Type II (beta) errors in the hand literature: the importance of power.J Hand Surg [Am]. 1998; 23: 20-25
- Type II error in the spine surgical literature.Spine. 2004; 29: 1146-1149
- The beta error and sample size determination in clinical trials in emergency medicine.Ann Emerg Med. 1987; 16: 183-187
- Beta, or type II error in psychiatric controlled clinical trials.J Psychiatr Res. 1985; 19: 563-567
- The other side of statistical significance: a review of type II errors in the Australian medical literature.Aust N Z J Med. 1982; 12: 7-9
- Low power, type II errors, and other statistical problems in recent cardiovascular research.Am J Physiol. 1997; 273: H487-493
- Negative results of randomized clinical trials published in the surgical literature: equivalency or error?.Arch Surg. 2001; 136: 796-800
- Empowering research: statistical power in general practice research.Fam Pract. 1997; 14: 324-329
- Sample size and statistical power in clinical orthopaedic research.J Bone Joint Surg Am. 1999; 81: 1454-1460
- Type-II error rates (beta errors) of randomized trials in orthopaedic trauma.J Bone Joint Surg Am. 2001; 83: 1650-1655
- The problem of the type II statistical error.Obstet Gynecol. 1995; 86: 857-859
- Statistical power, sample size, and their reporting in randomized controlled trials.JAMA. 1994; 272: 122-124
- Hypothesis testing in neurosurgical trials.J Neurosurg. 1981; 54: 468-472
- Inadequate size of “negative” clinical trials in dermatology.Br J Dermatol. 1993; 128: 317-326
- Safety of medicines: a guide to detecting and reporting adverse drug reactions.World Health Organization, Geneva2002
- Comparison of beta-blockers, amiodarone plus beta-blockers, or sotalol for prevention of shocks from implantable cardioverter defibrillators: the OPTIC Study: a randomized trial.JAMA. 2006; 295: 165-171
- Comparison of fixed-dose weight-adjusted unfractionated heparin and low-molecular-weight heparin for acute treatment of venous thromboembolism.JAMA. 2006; 296: 935-942
- Effects of abatacept in patients with methotrexate-resistant active rheumatoid arthritis: a randomized trial.Ann Intern Med. 2006; 144: 865-876
- Intraperitoneal cisplatin and paclitaxel in ovarian cancer.N Engl J Med. 2006; 354: 34-43
- Clomiphene, metformin, or both for infertility in the polycystic ovary syndrome.N Engl J Med. 2007; 356: 551-566
- Oral melphalan and prednisone chemotherapy plus thalidomide compared with melphalan and prednisone alone in elderly patients with multiple myeloma: randomised controlled trial.Lancet. 2006; 367: 825-831
- Estimating live birth rates after ovulation induction in polycystic ovary syndrome: sample size calculations for the pregnancy in polycystic ovary syndrome trial.Contemp Clin Trials. 2005; 26: 271-280
- Categorical data analysis.John Wiley and Sons, Inc., New York, NY2002
- The Wilcoxon, Spearman, Fisher, #2-, student and Pearson tests and 2x2 tables.Statistician. 1995; 44: 441-450
National Cancer Institute. Common toxicity criteria v.2. NCI; 1999 [updated 1999. Available at: http://ctep.cancer.gov/forms/CTCv20_4-30-992.pdf. [Accessed 2008 February 20]. April 30, 1999;
Christensen R, Kristensen PK, Bartels EM, Bliddal H, Astrup A. Efficacy and safety of the weight-loss drug rimonabant: a meta-analysis of randomised trials. Lancet; 370(): 1706–1713.
- Anti-TNF antibody therapy in rheumatoid arthritis and the risk of serious infections and malignancies: systematic review and meta-analysis of rare harmful effects in randomized controlled trials.JAMA. 2006; 295: 2275-2285
- Effect of rosiglitazone on the risk of myocardial infarction and death from cardiovascular causes.N Engl J Med. 2007; 356: 2457-2471
- Advances in risk-benefit evaluation using probabilistic simulation methods: an application to the prophylaxis of deep vein thrombosis.J Clin Epidemiol. 2004; 57: 795-803
- Sifting the evidence{–}what's wrong with significance tests? Another comment on the role of statistical methods.BMJ. 2001; 322: 226-231
- Reporting of adverse drug reactions in randomised controlled trials—a systematic survey.BMC Clin Pharmacol. 2001; 1: 3
Article info
Publication history
Published online: November 17, 2008
Accepted:
August 18,
2008
Identification
Copyright
© 2009 Elsevier Inc. Published by Elsevier Inc. All rights reserved.
