The conclusions on the effectiveness of homeopathy highly depend on the set of analyzed trials

      Abstract

      Objective

      Shang's recently published meta-analysis on homeopathic remedies (Lancet) based its main conclusion on a subset of eight larger trials out of 21 high quality trials (out of 110 included trials). We performed a sensitivity analysis on various other meaningful trial subsets of all high quality trials.

      Study Design

      Subsets were defined according to sample size, type of homeopathy, type of publication, and treated disease/condition. For each subset, we estimated the overall odds ratios (ORs) from random effect meta-analyses.

      Results

      All trials were highly heterogeneous (I2=62.2%). Homeopathy had a significant effect beyond placebo (OR=0.76; 95% CI: 0.59–0.99; p=0.039). When the set of analyzed trials was successively restricted to larger patient numbers, the ORs varied moderately (median: 0.82, range: 0.71–1.02) and the P-values increased steadily (median: 0.16, range: 0.03–0.93), including Shang's results for the eight largest trials (OR=0.88, CI: 0.66–1.18; P=0.41).
      Shang's negative results were mainly influenced by one single trial on preventing muscle soreness in 400 long-distance runners.

      Conclusions

      The meta-analysis results change sensitively to the chosen threshold defining large sample sizes. Because of the high heterogeneity between the trials, Shang's results and conclusions are less definite than had been presented.

      Keywords

      To read this article in full you will need to make a payment

      References

        • Shang A.
        • Huwiler-Muntener K.
        • Nartey L.
        • Juni P.
        • Dorig S.
        • Sterne J.A.
        • et al.
        Are the clinical effects of homoeopathy placebo effects? Comparative study of placebo-controlled trials of homoeopathy and allopathy.
        Lancet. 2005; 366: 726-732
        • Linde K.
        • Scholz M.
        • Ramirez G.
        • Clausius N.
        • Melchart D.
        • Jonas W.B.
        Impact of study quality on outcome in placebo-controlled trials of homeopathy.
        J Clin Epidemiol. 1999; 52: 631-636
        • Fisher P.
        • Berman B.
        • Davidson J.
        • Reilly D.
        • Thompson T.
        Are the clinical effects of homoeopathy placebo effects?.
        Lancet. 2005; 366 ([letter]): 2082-2083
      1. (4.2.4 [updated March 2005])Higgins J. Green S. Cochrane handbook for systematic reviews of interventions. John Wiley & Sons, Ltd., Chichester, UK2005
        • Thompson S.G.
        Why and how sources of heterogeneity should be investigated.
        in: Egger M. Smith G.D. Altman D. Systematic Reviews in Health Care. BMJ Publishing Group, London2001: 157-175
        • Rottey E.E.D.
        • Verleye G.B.
        • Liagre R.L.P.
        Het effect van een homeopathische bereiding van micro-organismen bij de preventie von griepsymtomen—Een gerandomiseerd dubbel-blind onderzoek in de huisartspraktijk.
        Tijdschr Integ Geneeskunde. 1995; 11: 54-58
        • Vickers A.J.
        • Fisher P.
        • Wyllie S.E.
        • Rees R.
        Homeopathic Arnica 30X is ineffective for muscle soreness after long-distance running—a randomized, double-blind, placebo-controlled trial.
        Clin J Pain. 1998; 14: 227-231
        • Papp R.
        • Schuback G.
        • Beck E.
        • Burkard G.
        • Bengel J.
        • Lehrl S.
        • et al.
        Oscillococcinum in patients with influenza-like syndromes: a placebo controlled double-blind evaluation.
        Br Homeopath J. 1998; 87: 69-76
        • Schmidt J.M.
        • Ostermayr B.
        Does a homeopathic ultramolecular dilution of Thyroidinum 30CH affect the decrease of body weight reduction in fasting patients?—A randomised placebo-controlled double-blind trial.
        Homeopathy. 2002; 91: 197-206
        • Labrecque M.
        • Audet D.
        • Latulippe L.
        • Drouin J.
        Homeopathic treatment of plantar warts.
        Can Med Assoc J. 1992; 146: 1749-1753
        • Jacobs J.
        • Jiménez L.M.
        • Malthouse S.
        • Chapman E.
        • Crothers D.
        • Masuk M.
        • et al.
        Homeopathic treatment of acute childhood diarrhea—results from a clinical trial in Nepal.
        J Alternat Complement Med. 2000; 6: 131-139
        • Weiser M.
        • Clasen B.P.E.
        Randomisierte plazebokontrollierte Doppelblindstudie zur Untersuchung der klinischen Wirksamkeit der homöopathischen Euphorbium compositum-Nasentropfen S bei chronischer Sinusitis.
        Forsch Komplementärmed. 1994; 1: 251-259
        • Walach H.
        • Haeusler W.
        • Lowes T.
        • Mussbach D.
        • Schamell U.
        • Springer W.
        • et al.
        Classical homeopathic treatment of chronic headaches.
        Cephalalgia. 1997; 17: 119-126
        • Jacobs J.
        • Jiménez L.M.
        • Gloyd S.S.
        • Gale J.L.
        • Crothers D.
        Treatment of acute childhood diarrhea with homoeopathic medicine: a randomized clinical trial in Nicaragua.
        Pediatrics. 1994; 93: 719-725
        • Jacobs J.
        • Springer D.A.
        • Crothers D.
        Homeopathic treatment of acute otitis media in children: a preliminary randomized placebo-controlled trial.
        Pediatr Infect Dis J. 2001; : 177-183
        • Hart O.
        • Mullee M.A.
        • Lewith G.
        • Miller J.
        Double-blind, placebo-controlled, randomized clinical trial of homoeopathic arnica C30 for pain and infection after total abdominal hysterectomy.
        J Royal Soc Med. 1997; 90: 73-78
        • Wiesenauer M.
        • Häussler S.
        • Gaus W.
        Pollinosis-Therapie mit Galphimia glauca.
        MMW Fortschr Med. 1983; 101: 811-814
        • Zell J.
        • Connert W.D.
        • Mau J.
        • Feuerstake G.
        Behandlung von akuten Sprunggelenksdistorsionen—Doppelblindstudie zum Wirksamkeitsnachweis eines homöopathischen Salbenpräparats.
        MMW Fortschr Med. 1988; 106: 96-100
        • Böhmer D.
        • Ambrus P.
        Behandlung von Sportverletzungen mit Traumeel-Salbe—Kontrollierte Doppelblindstudie.
        Biol Medizin. 1992; 21: 260-268
        • Vickers A.J.
        • Fisher P.
        • Smith C.
        • Wyllie S.E.
        • Lewith G.T.
        Homoeopathy for delayed onset muscle soreness—a randomised double blind placebo controlled trial.
        Brit J Sports Med. 1997; 31: 304-307
        • Jawara N.
        • Lewith G.T.
        • Vickers A.
        • Mullee M.A.
        • Smith C.
        Homoeopathic Arnica and Rhus toxicodendron for delayed onset muscle soreness—a pilot for a randomized, double-blind, placebo-controlled trial.
        Brit Hom J. 1997; 86: 10-15
        • Chapman E.H.
        • Weintraub R.J.
        • Milburn M.A.
        Homeopathic treatment of mild traumatic brain injury—a randomised, double-blind, placebo-controlled clinical trial.
        J Head Trauma Rehabil. 1999; 14: 521-542
        • Tveiten D.
        • Bruset S.
        • Borchgrevink C.F.
        • Norseth J.
        Effects of the homeopathic remedy Arnica D30 on marathon runners: a randomized, double-blind study during the 1995 Oslo Marathon.
        Complem Ther Med. 1998; 6: 71-74
        • Stevinson C.
        • Devaraj V.S.
        • Fountain-Barber A.
        Homeopathic arnica for prevention of pain and bruising: randomized placebo-controlled trial in hand surgery.
        J R Soc Med. 2003; 96: 60-65
        • Lepaisant C.
        Essai thérapeutique en homéopathie: traitement des tensions mammaires et mastodynies du syndrome prémenstruel.
        Rev Fr Gynécol Obstét. 1995; 90: 94-97
        • Chapman E.H.
        • Angelica J.
        • Spitalny G.
        • Strauss M.
        Results of a study of the homoeopathic treatment of PMS.
        J Am Inst Hom. 1994; 87: 14-21
        • Egger M.
        Study characteristics of homoeopathy studies.
        (Available from:) ([cited 06.03.2006])
        • Hasselblad V.
        • Hedges L.
        Meta-analysis of screening and diagnostic tests.
        Psychol Bull. 1995; 117: 167-178
        • Egger M.
        • Smith G.D.
        • Schneider M.
        • Minder C.
        Bias in meta-analysis detected by a simple, graphical test.
        British Med J. 1997; 315: 629-634
        • Thompson S.G.
        • Sharp S.J.
        Explaining heterogeneity in meta-anaysis: a comparison of methods.
        Statist Med. 1999; 18: 2693-2708
        • Fleiss J.
        The statistical basis of meta-analysis.
        Stat Meth Med Res. 1993; 2: 121-145
        • Higgins J.P.
        • Thompson S.G.
        Quantifying heterogeneity in a meta-analysis.
        Stat Med. 2002; 21: 1539-1558
        • Ernst E.
        • Barnes J.
        Are homoeopathic remedies effective for delayed-onset muscle soreness?—a systematic review of placebo-controlled trials.
        Perfusion. 1998; 11: 4-8
        • Linde K.
        • Jonas W.
        Are the clinical effects of homoeopathy placebo effects?.
        Lancet. 2005; 366 ([letter]): 2081-2082
        • Shang A.
        • Jüni P.
        • Sterne J.A.C.
        • Huwiler-Müntener K.
        • Egger M.
        Author's reply.
        Lancet. 2005; 366: 2083-2085
        • Egger M.
        • Smith G.D.
        • Phillips A.N.
        Meta analysis: principles and procedures.
        BMJ. 1997; 315: 1533-1537
        • Higgins J.P.T.
        • Thompson S.G.
        • Deeks J.J.
        • Altman D.G.
        Measuring inconsistency in meta-analyses.
        Br Med J. 2003; 327: 557-560
        • McAuley L.
        • Pham B.
        • Tugwell P.
        • Moher D.
        Does the inclusion of grey literature influence estimates of intervention effectiveness reported in meta-analyses?.
        Lancet. 2000; 356: 1228-1231
        • Chan A.
        • Hjobartsson A.
        • Haahr M.
        • Gotzsche P.
        • Altman D.
        Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles.
        JAMA. 2004; 291: 2457-2465
        • Pham B.
        • Klassen T.P.
        • Lawson M.L.
        • Moher D.
        Language of publication restrictions in systematic reviews gave different results depending on whether the intervention was conventional or complementary.
        J Clin Epidemiol. 2005; 58: 769-776
        • Moher D.
        • Fortin P.
        • Jadad A.R.
        • Juni P.
        • Klassen T.
        • Le Lorier J.
        • et al.
        Completeness of reporting of trials published in languages other than English: implications for conduct and reporting of systematic reviews.
        Lancet. 1996; 347: 363-366
        • Klassen T.P.
        • Pham B.
        • Lawson M.L.
        • Moher D.
        For randomized controlled trials, the quality of reports of complementary and alternative medicine was as good as reports of conventional medicine.
        J Clin Epidemiol. 2005; 58: 763-768
        • Linde K.
        • Clausius N.
        • Ramirez G.
        • Melchart D.
        • Eitel F.
        • Hedges L.
        • et al.
        Are the clinical effects of homoeopathy placebo effects?—a meta-analysis of placebo-controlled trials.
        Lancet. 1997; 350: 834-843
        • Dean M.E.
        The trials of homeopathy.
        KVC-Verlag, Essen2004
        • de Lange-de-Klerk E.S.M.
        • Blommers J.
        • Kuik D.J.
        • Bezemer P.D.
        • Feenstra L.
        Effects of homoeopathic medicines on daily burden of symptoms in children with recurrent upper respiratory tract infections.
        BMJ. 1994; 309: 1329-1332
        • Reilly D.T.
        • Taylor M.A.
        • Beattie N.G.M.
        • Campbell J.H.
        • McSharry C.
        • Aitchison T.C.
        • et al.
        Is evidence for homoeopathy reproducible?.
        Lancet. 1994; 344: 1601-1606
        • Reilly D.T.
        • Taylor M.A.
        • McSharry C.
        • Aitchison T.
        Is homoeopathy a placebo response?—Controlled trial of homoeopathic potency—with pollen in hayfever as model.
        Lancet. 1986; 2: 881-886
        • Hofmeyr G.J.
        • Piccioni V.
        • Blauhof P.
        Postpartum homoeopathic Arnica montana—a potency-finding pilot study.
        Br J Clin Pract. 1990; 44: 619-621
        • Vandenbroucke J.P.
        • de Craen A.J.
        Alternative medicine: a “mirror image” for scientific reasoning in conventional medicine.
        Ann Intern Med. 2001; 135: 507-513
        • Quine W.V.
        Ontological relativity and other essays.
        Columbia University Press, New York1969
        • Vandenbroucke J.P.
        Homoeopathy and “the growth of truth.
        Lancet. 2005; 366: 691-692
      2. Gezondheidsraad. Alternatieve Behandelwijzen en Wetenschappelijk Onderzoek [Alternative treatments and scientific research]. Den Haag; 1993.