The conclusions on the effectiveness of homeopathy highly depend on the set of analyzed trials

Abstract 

Objective

Shang's recently published meta-analysis on homeopathic remedies (Lancet) based its main conclusion on a subset of eight larger trials out of 21 high quality trials (out of 110 included trials). We performed a sensitivity analysis on various other meaningful trial subsets of all high quality trials.

Study Design

Subsets were defined according to sample size, type of homeopathy, type of publication, and treated disease/condition. For each subset, we estimated the overall odds ratios (ORs) from random effect meta-analyses.

Results

All trials were highly heterogeneous (I²=62.2%). Homeopathy had a significant effect beyond placebo (OR=0.76; 95% CI: 0.59–0.99; p=0.039). When the set of analyzed trials was successively restricted to larger patient numbers, the ORs varied moderately (median: 0.82, range: 0.71–1.02) and the P-values increased steadily (median: 0.16, range: 0.03–0.93), including Shang's results for the eight largest trials (OR=0.88, CI: 0.66–1.18; P=0.41).

Shang's negative results were mainly influenced by one single trial on preventing muscle soreness in 400 long-distance runners.

Conclusions

The meta-analysis results change sensitively to the chosen threshold defining large sample sizes. Because of the high heterogeneity between the trials, Shang's results and conclusions are less definite than had been presented.

Keywords: Homeopathy, Randomized clinical trials, Meta-analysis, Selection bias, Heterogeneity, Sensitivity analysis