Impact of Study Quality on Outcome in Placebo-Controlled Trials of Homeopathy


      We investigated the influence of indicators of methodological quality on study outcome in a set of 89 placebo-controlled clinical trials of homoeopathy in three different ways: (1) The results of studies meeting single criteria (explicit statement of random allocation, allocation concealment, double-blinding, completeness of follow-up) of methodological quality were compared with those of studies not meeting the criteria in univariate and multivariate analyses; (2) The results of studies scoring above and below predefined scores in two quality assessment scales were compared; (3) Primary studies were consecutively entered into a cumulative meta-analysis according to the summary scores derived from the quality assessment scales. All analyses were performed using meta-regression methods. Studies that were explicitly randomized and were double-blind as well as studies scoring above the cut-points yielded significantly less positive results than studies not meeting the criteria. In the cumulative meta-analyses, there was a trend for increasing effect sizes when more studies with lower-quality scores were added. However, there was no linear relationship between quality scores and study outcome. We conclude that in the study set investigated, there was clear evidence that studies with better methodological quality tended to yield less positive results. Because summarizing disparate study features into a single score is problematic, meta-regression methods simultaneously investigating the influence of single study features seem the best method for investigating the impact of study quality on outcome.


      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


      Subscribe to Journal of Clinical Epidemiology
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


        • Schulz K.F.
        • Chalmers I.
        • Hayes R.J.
        • Altman D.G.
        Empirical evidence of bias.
        JAMA. 1995; 273: 408-412
        • Khan K.S.
        • Daya S.
        • Jadad A.R.
        The importance of quality of primary studies in producing unbiased systematic reviews.
        Arch Intern Med. 1996; 156: 661-666
        • Moher D.
        • Pham B.
        • Jones A.
        • Cook D.J.
        • Jadad A.R.
        • Moher M.
        • et al.
        Does quality of reports of randomised trials affect estimates of intervention efficacy reported in meta-analyses?.
        Lancet. 1998; 352: 609-613
        • Moher D.
        • Jadad A.R.
        • Nichol G.
        • Penman M.
        • Tugwell P.
        • Walsh S.
        Assessing the quality of randomized controlled trials.
        Control Clin Trials. 1995; 16: 62-73
        • Greenland S.
        Quality scores are useless and potentially misleading.
        Am J Epidemiol. 1994; 140: 300-301
        • Olkin I.
        Invited commentary.
        Am J Epidemiol. 1994; 140: 297-299
        • Linde K.
        • Clausius N.
        • Ramirez G.
        • Melchart D.
        • Eitel F.
        • Hedges L.V.
        • et al.
        Are the clinical effects of homoeopathy placebo effects? A meta-analysis of placebo-controlled trials.
        Lancet. 1997; 350: 834-843
        • Jadad A.R.
        • Moore R.A.
        • Carrol D.
        • Jenkinson C.
        • Reynolds D.J.M.
        • Lavaghan D.J.
        • et al.
        Assessing the quality of reports of randomized clinical trials.
        Control Clin Trials. 1996; 17: 1-12
        • Linde K.
        • Worku F.
        • Stör W.
        • Wiesner-Zechweister M.
        • Pothenceuu R.
        • Weinschitz T.
        Randomized clinical trials of acupuncture for asthma—a systematic review.
        Fortschr Komplementärmed. 1996; 3: 148-155
        • Linde K.
        • Ramirez G.
        • Mulrow C.D.
        • Pauls A.
        • Weidenhammer W.
        • Melchart D.
        St John’s wort for depression—an overview and meta-analysis of randomised clinical trials.
        BMJ. 1996; 313: 253-258
        • DerSimonian R.
        • Laird N.
        Meta-analysis in clinical trials.
        Control Clin Trials. 1986; 7: 177-188
        • Berkey C.S.
        • Hoaglin D.C.
        • Mosteller F.
        • Colditz G.A.
        A random-effects regression model for meta-analysis.
        Stat Med. 1995; 14: 395-411
        • Laird N.M.
        • Ware J.R.
        Random-effects models of longitudinal data.
        Biometrics. 1982; 38: 963-974
        • Walach H.
        • Haeusler W.
        • Lowes T.
        • Mussbach D.
        • Schawell U.
        • Springer W.
        Classical homeopathic treatment of chronic headache.
        Cephalalgia. 1997; 17: 119-126
        • Whitmarsh T.E.
        • Coleston-Shields D.M.
        • Steiner T.J.
        Double-blind randomized placebo-controlled study of homoeopathic prophylaxis of migraine.
        Cephalalgia. 1997; 17: 600-604
        • Linde K.
        • Melchart D.
        Randomized controlled trials of individualized homeopathy—a state-of-the-art review.
        J Alt Complement Med. 1998; 4: 371-388
        • Chalmers T.C.
        • Smith H.
        • Blackburn B.
        • Silverman B.
        • Schroeder B.
        • Reitman D.
        A method for assessing the quality of a randomized control trial.
        Control Clin Trials. 1981; 2: 31-39
        • Emerson J.D.
        • Burdick E.
        • Hoaglin D.C.
        • Mosteller F.
        • Chalmers T.C.
        An empirical study of the possible relation of treatment differences to quality scores in controlled randomized clinical trials.
        Control Clin Trials. 1990; 11: 339-352
        • Greenland S.
        in: Rothman K. Greenland S. Modern Epidemiology. Lippincott Raven, Philadelphia, PA1998
        • Moher D.
        • Jadad A.R.
        • Tugwell P.
        Assessing the quality of randomized controlled trials. Current issues and future directions.
        Int J Technol Assessment Health Care. 1996; 12: 195-208
        • Nurmohamed M.T.
        • Rosendaal F.R.
        • Bueller H.R.
        • Dekker E.
        • Howmes D.W.
        • Vandenbroucke J.P.
        Low-molecular weight heparin versus standard heparin in general and orthopedic surgery.
        Lancet. 1992; 340: 152-156
        • Begg C.
        • Cho M.
        • Eastwood S.
        • Horton R.
        • Moher D.
        • Olkin J.
        Improving the quality of reporting of randomized trials.
        JAMA. 1996; 276: 637-639