Abstract
Objectives
The hierarchy of evidence presupposes linearity and additivity of effects, as well
as commutativity of knowledge structures. It thereby implicitly assumes a classical
theoretical model.
Study Design and Setting
This is an argumentative article that uses theoretical analysis based on pertinent
literature and known facts to examine the standard view of methodology.
Results
We show that the assumptions of the hierarchical model are wrong. The knowledge structures
gained by various types of studies are not sequentially indifferent, that is, do not
commute. External validity and internal validity are at least partially incompatible
concepts. Therefore, one needs a different theoretical structure, typical of quantum-type
theories, to model this situation. The consequence of this situation is that the implicit
assumptions of the hierarchical model are wrong, if generalized to the concept of
evidence in total.
Conclusion
The problem can be solved by using a matrix-analytical approach to synthesizing evidence.
Here, research methods that produce different types of evidence that complement each
other are synthesized to yield the full knowledge. We show by an example how this
might work. We conclude that the hierarchical model should be complemented by a broader
reasoning in methodology.
Key Words
To read this article in full you will need to make a payment
Purchase one-time access:
Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online accessOne-time access price info
- For academic or personal research use, select 'Academic and Personal'
- For corporate R&D use, select 'Corporate R&D Professionals'
Subscribe:
Subscribe to Journal of Clinical EpidemiologyAlready a print subscriber? Claim online access
Already an online subscriber? Sign in
Register: Create an account
Institutional Access: Sign in to ScienceDirect
References
- Evidence based medicine: how to practice and teach EBM.Churchill Livingstone, New York1997
- Randomization, balance, and the validity and efficiency of design-adaptive allocation methods.J Stat Plan Inference. 2001; 94: 97-119
- Randomized versus historical controls for clinical trials.Am J Med. 1982; 72: 233-240
- Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials.JAMA. 1995; 273: 408-412
- Detecting selection bias in randomized clinical trials.Control Clin Trials. 1999; 20: 319-327
- Risks and benefits of estrogen plus progestin in healthy postmenopausal women: principal results from the Women's Health Initiative randomized controlled trial.JAMA. 2002; 288: 321-333
- Bias in treatment assignment in controlled clinical trials.N Engl J Med. 1983; 309: 1358-1361
- Avoidance of large biases and large random errors in the assessment of moderate treatment effects: the need for systematic overviews.Stat Med. 1987; 6: 245-250
- Meta-analysis of clinical trials as a scientific discipline. I: control of bias and comparison with large co-operative trials.Stat Med. 1987; 6: 315-325
- The philosophy of evidence-based medicine.Wiley-Blackwell, Chichester2011
- GRADE: an emerging consensus on rating quality of evidence and strength of recommendations.BMJ. 2008; 336: 924
- External validity of randomised controlled trials: “to whom do the results of this trial apply?”.Lancet. 2005; 365: 82-93
- The mythology of randomization.in: Abel U. Koch A. Nonrandomized comparative clinical studies. Symposion Publishing, Düsseldorf1998: 27-40
- Why we need observational studies to evaluate the effectiveness of health care.BMJ. 1996; 312: 1215-1218
- Can randomised trials inform clinical decisions about individual patients?.Lancet. 1999; 353: 743-746
- Quasi-experimentation design and analysis issues for field settings.Rand McNally, Chicago1979
- Evaluating preference effects in partially unblinded randomized clinical trials.J Clin Epidemiol. 2003; 56: 109-115
- Attrition in substance abuse comparative treatment research: the illusion of randomization.in: Onken L.S. Blaine J.D. Psychotherapy and counseling in the treatment of drug abuse. National Institute of Drug Abuse, Rockville1990: 66-79
- Patients' willingness to enter clinical trials: measuring the association with perceived benefit and preference for decision participation.Soc Sci Med. 1991; 32: 35-42
- Patient preferences and randomized clinical trials.BMJ. 1989; 299: 313-315
- Problems of randomized trials.in: Abel U. Koch A. Nonrandomized comparative clinical studies. Symposion Publishing, Düsseldorf1998: 1-13
- Why are patients in clinical trials of heart failure not like those we see in everyday practice?.J Clin Epidemiol. 2003; 56: 1157-1162
- How expert physicians would wish to be treated if they had genitourinary cancer.J Clin Oncol. 1988; 6: 1736-1745
- The generalizability of psychotherapy efficacy trials in major depressive disorder: an analysis of the influence of patient selection in efficacy trials on symptom outcome in daily practice.BMC Psychiatry. 2012; 12: 192
- Eligibility criteria of randomized controlled trials published in high-impact general medical journals.JAMA. 2007; 297: 1233-1240
- Acute and longer-term outcomes in depressed outpatients requiring one or several treatment steps: a STAR*D report.Am J Psychiatry. 2006; 163: 1905-1917
- Efficacy and effectiveness of antidepressants: current status of research.Psychother Psychosom. 2010; 79: 267-279
- The road to recovery from depression—don't drive today with yesterday's map.Psychother Psychosom. 2007; 76: 260-265
- Equipoise and the ethics of randomization.J R Soc Med. 1995; 88: 552-559
- Equipoise and the dilemma of randomized clinical trials.N Engl J Med. 2011; 364: 476-480
- Experiences in the realisation of a research project on anthroposophical medicine in patients with advanced cancer.Schweiz Med Wochenschr. 2000; 130: 1173-1184
- In randomization we trust? There are overlooked problems in experimenting with people in behavioral intervention trials.J Clin Epidemiol. 2014; 67: 247-253
- Meditation programs for psychological stress and well-being: a systematic review and meta-analysis.JAMA. 2014; 174: 357-368
- Meditation is no medication (Letter).JAMA Intern Med. 2014; 17: 1193-1194
- Double-blind: double talk or are there ways to do better research.Med Hypotheses. 1986; 21: 119-126
- When are randomised trials unnecessary? Picking signal from noise.BMJ. 2007; 334: 349-351
- Researching complementary and alternative treatments—the gatekeepers are not at home.BMC Med Res Methodol. 2007; 7: 7
- Are treatments more effective than placebos? A systematic review and meta-analysis.PLoS One. 2013; 8: e62599
- Placebo effect in double-blind clinical trials: a review of interactions with medications.Lancet. 1994; 344: 1347-1349
- The therapeutic effect of clinical trials: understanding placebo response rates in clinical trials—a secondary analysis.BMC Med Res Methodol. 2005; 5: 26
- Meissner K. Kohls N. Colloca L. Placebo effects in medicine: mechanisms and clinical implications. Royal Society, London2011
- A comparison of placebo effects in clinical analgesic trials versus studies of placebo analgesia.Pain. 2002; 99: 443-452
- Is the placebo powerless? An analysis of clinical trials comparing placebo with no treatment.N Engl J Med. 2001; 344: 1594-1602
- Components of placebo effect: randomised controlled trial in patients with irritable bowel syndrome.BMJ. 2008; 336: 999-1003
- Deconstructing the placebo effect and finding the meaning response.Ann Intern Med. 2002; 136: 471-476
- Placebo controls: historical, methodological and general aspects.Philos Trans R Soc Lond B Biol Sci. 2011; 366: 1870-1878
- Circular instead of hierarchical—methodological principles for the evaluation of complex interventions.BMC Med Res Methodol. 2006; 6: 29
- The efficacy paradox in randomized controlled trials of CAM and elsewhere: beware of the placebo trap.J Altern Complement Med. 2001; 7: 213-218
- Selective publication of antidepressant trials and its influence on apparent efficacy.N Engl J Med. 2008; 358: 252-260
- Initial severity and antidepressant benefits: a meta-analysis of data submitted to the food and drug administration.PLoS Med. 2008; 5: e45
- Meta-analysis of the placebo response in antidepressant trials.J Affect Disord. 2009; 118: 1-8
- Acupuncture and knee osteoarthritis.Ann Intern Med. 2006; 145: 12-20
- German Acupuncture Trials (GERAC) for chronic low back pain: randomized, multicenter, blinded, parallel-group trial with 3 groups.Arch Intern Med. 2007; 167: 1892-1898
- Efficacy of acupuncture for the prophylaxis of migraine: a multicentre randomised controlled clinical trial.Lancet Neurol. 2006; 5: 310-316
- Donepezil and memantine for moderate-to-severe Alzheimer's disease.N Engl J Med. 2012; 366: 893-903
- Influence of psychological coping on survival and recurrence in people with cancer: systematic review.BMJ. 2002; 325: 1066
- Meta-analysis of the effects of psychosocial interventions on survival time in cancer patients.J Psychosom Res. 2004; 57: 123-131
- Systematically reviewing qualitative and quantitative evidence to inform management and policy-making in the health field.J Health Serv Res Policy. 2005; 10: 6-20
- Evidence-based policy: in search of a method.Evaluation. 2002; 8: 157-181
- Synthesizing evidence on complex interventions: how meta-analytical, qualitative, and mixed-method approaches can contribute.J Clin Epidemiol. 2013; 66: 1230-1243
- Dealing with complex causality in realist synthesis: the promise of qualitative comparative analysis.Am J Eval. 2012; 33: 60-78
- Protocol—realist and meta-narrative evidence synthesis: evolving standards (RAMESES).BMC Med Res Methodol. 2011; 11: 115
- A research and development agenda for systematic reviews that ask complex questions about complex interventions.J Clin Epidemiol. 2013; 66: 1262-1270
- Weak quantum theory: complementarity and entanglement in physics and beyond.Foundations Phys. 2002; 32: 379-406
- Weak quantum theory: formal framework and selected applications.in: Khrennikov A. Quantum theory: reconsiderations of foundations—American Institute of Physics, Conference Proceedings. Melville, New York, USA2006
- Generalized quantum theory: overview and latest developments.Axiomathes. 2011; 21: 211-220
- Generalizing quantum theory—approaches and applications.Axiomathes. 2011; 21: 185-371
- Generalised quantum theory—basic idea and general intuition: a background story and overview.Axiomathes. 2011; 21: 185-209
- Can quantum probability provide a new direction for cognitive modeling.Behav Brain Sci. 2013; 36: 255-327
- Hierarchy of evidence: a framework for ranking evidence evaluating healthcare interventions.J Clin Nurs. 2003; 12: 77-84
- The concept of complementarity and its role in quantum entanglement and generalized entanglement.Axiomathes. 2013; 23: 443-459
- Causality and complementarity.Philos Sci. 1937; 4: 289-298
- Beyond randomised versus observational studies.Lancet. 2004; 363: 1660-1661
- Observational methods in comparative effectiveness research.Am J Med. 2010; 123: e16-e23
- Should systematic reviews include non-randomized and uncontrolled studies? The case of acupuncture for chronic headache.J Clin Epidemiol. 2002; 55: 77-85
- Complex interventions and their implications for systematic reviews: a pragmatic approach.J Clin Epidemiol. 2013; 66: 1209-1214
- The evidence profile. The multidimensional nature of proof.Complement Ther Med. 1993; 1: 11-12
- Building an evidence house: challenges and solutions to research in complementary and alternative medicine.Forsch Komplementrmed Klass Naturheilkd. 2005; 12: 159-167
- Matrix analysis as a complementary analytic strategy in qualitative inquiry.Qual Health Res. 2002; 12: 855-866
- The evidence that evidence-based medicine omits.Prev Med. 2013; 57: 745-747
- Using a healthcare matrix to assess patient care in terms of aims for improvement and core competencies.J Qual Patient Saf. 2005; 31: 98-105
- Lead (Pb) and the risk of Alzheimer's disease or cognitive decline: a systematic review.Toxin Rev. 2011; 30: 103-114
- Copper and iron in Alzheimer's disease: a systematic review and its dietary implications.Br J Nutr. 2012; 107: 7-19
- Fruit, vegetables, and prevention of cognitive decline or dementia: a systematic review of cohort studies.J Nutr Health Aging. 2012; 16: 626-630
- The omega-6/omega-3 ratio and dementia or cognitive decline: a systematic review on human studies and biological evidence.J Nutr Gerontol Geriatr. 2013; 32: 1-23
- Does inorganic mercury play a role in Alzheimer's disease? A systematic review and an integrated molecular mechanism.J Alzheimers Dis. 2010; 22: 357-374
- Reducing systematic reviews to a cut and paste.Forsch Komplementrmed. 2010; 17: 303-305
Article info
Publication history
Published online: July 03, 2015
Accepted:
March 23,
2015
Footnotes
Conflicts of interest: None
Funding: The funding was provided by the Samueli Institute for work around that concept. M.L. was supported by a grant from the Oberberg Foundation, Berlin, while working on this article.
Identification
Copyright
© 2015 Elsevier Inc. Published by Elsevier Inc. All rights reserved.
ScienceDirect
Access this article on ScienceDirectLinked Article
- The hierarchy of evidence and quantum theoryJournal of Clinical EpidemiologyVol. 72
- PreviewMedicine is complex, and evidence regarding the effects of interventions is subject to multiple sources of bias. The dominant view is that the “best available evidence” for the effects of interventions comes from randomized trials or systematic reviews of randomized trials. For instance, evidence-based medicine provides a hierarchy of evidence for assessing the benefits of treatments, which places well-conducted randomized trials and systematic reviews of randomized trials above other sources of evidence including observational studies.
- Full-Text
- Preview
- Portability of therapeutic effects (letter commenting: J Clin Epidemiol. 2015;68(11):1251--1260.)Journal of Clinical EpidemiologyVol. 75
- There is no reverse hierarchy (letter commenting: J Clin Epidemiol. 2015;68(11):1251--1260.)Journal of Clinical EpidemiologyVol. 75
- PreviewI am neither an expert in non-Abelian algebra nor in quantum theory, so I do not feel competent to judge the corresponding arguments of the authors [1]. The basis of these arguments, however, is obviously wrong. There is no reverse hierarchy of internal and external validity, as is suggested in Figure 1 and explicitly stated in the text: “The more internal validity is emphasized in a study, the lower external validity tends to become.”
- Full-Text
- Preview
