Investigating complexity in systematic reviews of interventions by using a spectrum of methods


      Systematic reviews framed by PICOS (Populations, Interventions, Comparisons, Outcomes, and Study designs) have been valuable for synthesizing evidence about the effects of interventions. However, this framework is limited in its utility for exploring the influence of variations within populations or interventions, or about the mechanisms of action or causal pathways thought to mediate outcomes, other contextual factors that might similarly moderate outcomes, or how and when these mechanisms and elements interact. Valuable insights into these issues come from configurative as well as aggregative methods of synthesis. This article considers the range of evidence that can be used in systematic reviews of interventions to investigate complexity in terms of potential sources of variation in interventions and their effects, and presents a continuum of purposes for, and approaches to, evidence synthesis. Choosing an appropriate synthesis method takes into account whether the purpose of the synthesis is to generate, explore, or test theories. Taking complexity into account in a synthesis of economic evidence similarly shifts emphasis from evidence synthesis strategies focused on aggregation toward configurative strategies that aim to develop, explore, and refine (in advance of testing) theories or explanations of how and why interventions are more or less resource intensive, costly or cost-effective in different settings, or when implemented in different ways.


      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


      Subscribe to Journal of Clinical Epidemiology
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


        • Naaldenberg J.
        • Vaandrager L.
        • Koelen M.
        • Wagemakers A.M.
        • Saan H.
        • de H.K.
        Elaborating on systems thinking in health promotion practice.
        Glob Health Promot. 2009; 16: 39-47
        • Craig P.
        • Dieppe P.
        • Macintyre S.
        • Michie S.
        • Nazareth I.
        • Petticrew M.
        Developing and evaluating complex interventions: new guidance.
        UK Medical Research Council, London2008
      1. Cochrane Collaboration. The Cochrane Library Available at: 2012. Accessed June 7, 2012.

        • Waters E.
        • Hall B.J.
        • Armstrong R.
        • Doyle J.
        • Pettman T.L.
        • de Silva-Sanigorski A.
        Essential components of public health evidence reviews: capturing intervention complexity, implementation, economics and equity.
        J Public Health (Oxf). 2011; 33: 462-465
        • Moher D.
        • Schulz K.F.
        • Altman D.G.
        The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomised trials.
        Lancet. 2001; 357: 1191-1194
        • Des Jarlais D.C.
        • Lyles C.
        • Crepaz N.
        Improving the reporting quality of nonrandomized evaluations of behavioral and public health interventions: the TREND statement.
        Am J Public Health. 2004; 94: 361-366
        • APA Publications and Communications Board Working Group on Journal Article Reporting Standards
        Reporting standards for research in psychology.
        Am Psychol. 2008; 63: 839-851
        • Flay B.R.
        • Biglan A.
        • Boruch R.F.
        • Castro F.G.
        • Gottfredson D.
        • Kellam S.
        • et al.
        Standards of evidence: criteria for efficacy, effectiveness and dissemination.
        Prev Sci. 2005; 6: 151-175
        • Squires J.
        • Valentine J.
        • Grimshaw J.
        Systematic reviews of complex interventions: framing the review question.
        J Clin Epidemiol. 2013; 66 ([in this issue]): 1215-1222
        • Popay J.
        Moving beyond effectiveness in evidence synthesis: methodological issues in the synthesis of diverse sources of evidence.
        National Institute for Health and Clinical Excellence, London, UK2006
        • Anderson L.
        • Petticrew M.
        • Rehfuess E.
        • Armstrong R.
        • Ueffing E.
        • Baker P.
        • et al.
        Using logic models to capture complexity in systematic reviews.
        Res Synth Methods. 2011; 2: 33-42
        • Turley R.
        • Saith R.
        • Bhan N.
        • Rehfuess E.
        • Carter B.
        Slum upgrading strategies involving physical environment and infrastructure interventions and their effects on health and socio-economic outcomes.
        Cochrane Database Syst Rev. 2012; 1 (CD010067)
        • Coren E.
        • Hossain R.
        • Pardo P.J.
        • Veras M.M.
        • Chakraborty K.
        • Harris H.
        • et al.
        Interventions for promoting reintegration and reducing harmful behaviour and lifestyles in street-connected children and young people.
        Cochrane Database Syst Rev. 2013; 2 (CD009823)
        • Ivers N.
        • Jamtvedt G.
        • Flottorp S.
        • Young J.
        • Odgaard-Jensen J.
        • French S.
        • et al.
        Audit and feedback: effects on professional practice and healthcare outcomes.
        Cochrane Database Syst Rev. 2012; 6 (CD000259)
        • Koehlmoos T.
        • Gazi R.
        • Hossain S.S.
        • Rashid M.
        Social franchising evaluations: a scoping review.
        EPPI-Centre, Social Science Research Unit, Institute of Education, University of London, London, UK2011
        • Koehlmoos T.P.
        • Gazi R.
        • Hossain S.S.
        • Zaman K.
        The effect of social franchising on access to and quality of health services in low- and middle-income countries.
        Cochrane Database Syst Rev. 2009; : CD007136
        • Stewart R.
        • van Rooyen C.
        • Dickson K.
        • Majoro M.
        • de Wet T.
        What is the impact of microfinance on poor people? A systematic review of evidence from sub-Saharan Africa.
        EPPI-Centre, Social Science Research Unit, University of London, London, UK2010
        • Petticrew M.
        • Rehfuess E.
        • Noyes J.
        • Noyes J.
        • Pantoja T.
        • Shemilt I.
        • et al.
        Synthesising evidence on complex interventions: the contribution of meta-analytic, qualitative and mixed-method approaches.
        J Clin Epidemiol. 2013; 66: 1230-1243
        • Kannampallil T.G.
        • Schauer G.F.
        • Cohen T.
        • Patel V.L.
        Considering complexity in healthcare systems.
        J Biomed Inform. 2011; 44: 943-947
        • Fewtrell L.
        • Colford Jr., J.M.
        Water, sanitation and hygiene in developing countries: interventions and diarrhoea—a review.
        Water Sci Technol. 2005; 52: 133-142
        • Gardner B.
        • Whittington C.
        • McAteer J.
        • Eccles M.P.
        • Michie S.
        Using theory to synthesise evidence from behaviour change interventions: the example of audit and feedback.
        Soc Sci Med. 2010; 70: 1618-1625
        • Gough D.
        • Oliver S.
        • Thomas J.
        Introduction to systematic reviews.
        Sage, London, UK2012
        • Haynes R.B.
        • Ackloo E.
        • Sahota N.
        • McDonald H.P.
        • Yao X.
        Interventions for enhancing medication adherence.
        Cochrane Database Syst Rev. 2008; : CD000011
        • Britten N.
        • Campbell R.
        • Pope C.
        • Donovan J.
        • Morgan M.
        • Pill R.
        Using meta ethnography to synthesise qualitative research: a worked example.
        J Health Serv Res Policy. 2002; 7: 209-215
        • Thomas J.
        • Harden A.
        • Oakley A.
        • Oliver S.
        • Sutcliffe K.
        • Rees R.
        • et al.
        Integrating qualitative research with trials in systematic reviews.
        BMJ. 2004; 328: 1010-1012
        • Oliver S.R.
        • Rees R.W.
        • Clarke-Jones L.
        • Milne R.
        • Oakley A.R.
        • Gabbay J.
        • et al.
        A multidimensional conceptual framework for analysing public involvement in health services research.
        Health Expect. 2008; 11: 72-84
        • Carroll C.
        • Booth A.
        • Cooper K.
        A worked example of “best fit” framework synthesis: a systematic review of views concerning the taking of some potential chemopreventive agents.
        BMC Med Res Methodol. 2011; 11: 29
        • Roberts K.A.
        • Dixon-Woods M.
        • Fitzpatrick R.
        • Abrams K.R.
        • Jones D.R.
        Factors affecting uptake of childhood immunisation: a Bayesian synthesis of qualitative and quantitative evidence.
        Lancet. 2002; 360: 1596-1599
        • Candy B.
        • King M.
        • Jones L.
        • Oliver S.
        Using qualitative synthesis to explore heterogeneity of complex interventions.
        BMC Med Res Methodol. 2011; 11: 124
        • Pawson R.
        • Greenhalgh T.
        • Harvey G.
        • Walshe K.
        Realist review—a new method of systematic review designed for complex policy interventions.
        J Health Serv Res Policy. 2005; 10: 21-34
      2. Booth A, Carroll C. Towards a transparent method for identifying theory for use in systematic reviews of behaviour change interventions: the BeHEMoTh procedure. Implement Sci. [In press.]

        • Hannes K.
        • Lockwood C.
        Synthesizing qualitative research: choosing the right approach.
        Wiley-Balckwell, Chichester, UK2012
        • Rees R.
        • Oliver S.
        Stakeholder perspectives and participation in reviews.
        in: Gough D. Oliver S. Thomas J. Introduction to systematic reviews. Sage, London, UK2012
        • Sculpher M.J.
        • Pang F.S.
        • Manca A.
        • Drummond M.F.
        • Golder S.
        • Urdahl H.
        • et al.
        Generalisability in economic evaluation studies in healthcare: a review and case studies.
        Health Technol Assess. 2004; 8 (1–192): iii-iiv
        • Anderson R.
        • Shemilt I.
        The role of economic perspectives and evidence in systematic review.
        in: Shemilt I. Mugford M. Vale L. Marsh K. Donaldson C. Evidence-based decisions and economics: health care, social welfare, education and criminal justice. Wiley-Blackwell, Oxford, UK2010