If you don't remember your password, you can reset it by entering your email address and clicking the Reset Password button. You will then receive an email that contains a secure link for resetting your password
If the address matches a valid account an email will be sent to __email__ with instructions for resetting your password
To identify examples of how social theories are used in systematic reviews of complex interventions to inform production of Cochrane guidance.
Study Design and Setting
Secondary analysis of published/unpublished examples of theories of social phenomena for use in reviews of complex interventions identified through scoping searches, engagement with key authors and methodologists supplemented by snowballing and reference searching. Theories were classified (low-level, mid-range, grand).
Results
Over 100 theories were identified with evidence of proliferation over the last 5 years. New low-level theories (tools, taxonomies, etc) have been developed for classifying and reporting complex interventions. Numerous mid-range theories are used; one example demonstrated how control theory had changed the review's findings. Review-specific logic models are increasingly used, but these can be challenging to develop. New low-level and mid-range psychological theories of behavior change are evolving. No reviews using grand theory (e.g., feminist theory) were identified. We produced a searchable Wiki, Mendeley Inventory, and Cochrane guidance.
Conclusions
Use of low-level theory is common and evolving; incorporation of mid-range theory is still the exception rather than the norm. Methodological work is needed to evaluate the contribution of theory. Choice of theory reflects personal preference; application of theory is a skilled endeavor.
Over 100 social theories that had been used or were designed for use in systematic reviews were identified with evidence of proliferation over the last 5 years.
•
New low-level theories (tools, taxonomies etc.) have been developed for classifying and reporting complex interventions.
•
Numerous mid-range theories are used; one example demonstrated how control theory had changed the review's findings.
•
Review-specific logic models are increasingly used, but these can be challenging to develop.
•
New low-level and mid-range psychological theories of behaviour change are evolving.
•
No reviews using grand theory (e.g. feminist theory) were identified.
What this adds to what was known?
•
Current systematic review guidance and methods manuals say little about use of social theories in complex intervention reviews; this is a major gap.
•
For the first time low-level, mid-range and grand theories are defined, classified and articulated in the context of systematic reviews of complex interventions.
•
New Cochrane guidance is provided on the selection of social theories in complex intervention reviews.
•
Two new searchable author resources (a ‘Theory in Reviews’ Wiki and Mendeley Theory in Reviews Inventory) are presented.
What is the implication and what should change now?
•
Use of appropriate theory can enhance and strenthen systematic review methods and interpretation of complex evidence.
•
Review authors are invited to use the Cochrane guidance and searchable resources when designing and conducting their reviews.
•
Choice of social theory reflects personal preference and application of theory in a systematic review is a skilled endeavour.
•
Review authors may benefit from additional professional development and training to make best use of social theories.
•
Methodological work is needed to further evaluate the contribution of social theory to systematic reviews of complex interventions.
1. Introduction
The importance and use of social theories in health and social care research has become increasingly evident over the last couple of decades. Alderson, in a seminal article published in the British Medical Journal in 1998, stated that “theories range from explicit hypotheses to working models and frameworks of thinking about reality” and that “the choice of theory, although often unacknowledged, shapes the way practitioners and researchers collect and interpret evidence” [
]. Reeves et al. expanded this idea by suggesting that “theories also provide complex and comprehensive conceptual understandings of things that cannot be pinned down: how societies work, how organizations operate, why people interact in certain ways” [
]. From a sociological perspective, Merton classified theories as low-level, mid-range, or grand theory lying on a spectrum “between the minor but necessary working hypotheses that evolve in abundance during day-to-day research and the all-inclusive systematic efforts to develop a unified theory that will explain all the observed uniformities of social behavior, social organization and social change” [
]. The boundaries between theory levels can however overlap and theories can transcend levels (or be refuted and discarded) as they are developed and tested over time. Merton's classification can be applied to theory used in systematic reviews as follows.
1.1 Low-level theory
Low-level theories (e.g., segregated hypotheses or isolated propositions, and typologies and taxonomies, etc) are used to predict, assume, describe, or organize aspects of the phenomena of interest but do not show the interrelationships between concepts. All reviews contain low-level theory in the form of segregated hypotheses or questions, but review designs and methods vary in the degree to which they incorporate recognized frameworks to systematize the review processes such as use of PICO [
]) have interconnected relationships between concepts with limited scope to explain specific phenomena, are empirically testable, and can be used to describe and predict causal relationships among concepts, or used to define activities and processes and predict outcomes. The Theory of Planned Behaviour, for example, is used to predict a person's intention to engage in a particular behavior at a specific time in a specific context. Some more sophisticated hypotheses can also be defined as mid-range theories. Similarly, “Programme theories” that make explicit the causal assumptions as to how a complex intervention is intended to work may start off as low-level theories and be developed into mid-range theory [
Grand theories are highly abstracted theories in which organized and integrated concepts explain the social world (e.g., Feminist theory, Welfarism, or Marxism). Feminist theory for example explains the phenomena of gender inequality in all social interactions at societal level, which distinguishes it as a grand theory because its focus moves beyond the more limited context of mid-range theory.
1.4 Evidence-based health care as a social theory
If defined as a social theory, evidence-based health care in its broadest sense (combination of best evidence [beyond the randomized controlled trial], patient/population perspective, and clinical judgment) could be conceptualized as a grand theory as well as a philosophy and scientific method underpinning decision making. Evidence-based health care evolved from the conceptually narrower evidence-based medicine which privileges the randomized controlled trial as the best form of evidence. Although Cochrane reviews contribute to evidence-based health care, in isolation, the standard Cochrane review of intervention effects is anchored within the positivist hierarchical epistemology of evidence-based medicine. As a consequence, it prioritizes aggregation of a limited number of predetermined primary and secondary outcomes from randomized controlled trials to explain a specific phenomenon of interest (intervention effect) which is more closely aligned to mid-range theory [
], and where theory can inform the design and conduct, and also be a product of systematic reviews. Application of social theory is common in the context of primary (especially qualitative) research to understand complex issues through specific “lenses” and to analyze and focus attention on different aspects of data [
]. A few review authors who use standard Cochrane review methods have however given explicit consideration to theory when evaluating included primary studies. The Cochrane Public Health Group recognizes in their supplemental guidance to the Cochrane Handbook that “as interventions become more multi-faceted, and thus more complex, it is important to reflect on the role theory has played” [
]. For example, in a systematic review of Internet-based interventions to promote health behavior change, Webb et al. found that theory-based interventions were more effective than those not based on theory [
Using the internet to promote health behaviour change: a systematic review and meta-analysis of the impact of theoretical basis, use of behaviour change techniques and mode of delivery on efficacy.
]. The limitation of the standard Cochrane approach is that beyond answering a simple question about intervention and effect, it cannot easily address complex questions or accommodate the synthesis of complex interventions with multiple causal pathways, interactions, and outcomes. Cochrane has however in recent times become more innovative and published nonstandard Cochrane reviews that integrate a synthesis of qualitative evidence to explain different intervention and implementation effects that more closely align with the broader evidence-based health care context [
Barriers and facilitators to the implementation of lay health worker programmes to improve access to maternal and child health: qualitative evidence synthesis.
]. The Cochrane Qualitative and Implementation Methods Group (http://cqim.cochrane.org/) has been a driving force behind repositioning Cochrane as a producer of mixed-method and qualitative evidence syntheses linked with Cochrane effect reviews that contribute to evidence-based health care decision making.
Fig. 1Spectrum and potential use of theory in the context of systematic reviews. Based on Merton's hierarchy of theory
]) are positioned within a realist epistemology and foreground theory use and development with different types of evidence as a way of understanding the complex world and multiple potential realities and outcomes. More recently, perhaps as a consequence of more theory-informed primary research and development of newer theory-led synthesis methods, the potential role of social theory (in particular low-level and mid-range) in Cochrane systematic reviews of complex interventions, or reviews where complexity is an important consideration, has captured increasing interest from review authors and methodologists alike. New United Kingdom (UK) Medical Research Council (MRC) guidance on the design and conduct of process evaluations alongside randomized controlled trials outlines the importance of using theory-informed methods to understand the functioning of a complex intervention [
]. It is therefore not unreasonable to anticipate that future trials of complex interventions are more likely to be designed with more sophisticated theory-informed process evaluations that produce various types of data and evidence amenable to synthesis that shed light on a range of short, medium, and longer term options and outcomes for decision makers to consider.
Although interest in theory in systematic review gathers pace, methods guidance, such as the Cochrane Handbook [
Higgins J.P.T. Green S. Cochrane handbook for systematic reviews of interventions Version 5.1.0 [updated March 2011]. The Cochrane Collaboration,
London, UK2011
] has a notable absence of reference to, or guidance on, the use of theory in reviews, other than commonly used low-level theory (e.g., frameworks and tools) to systematize the review process. Even then, many systematic reviewers would probably not recognize or conceptualize common systematic review frameworks and tools (PICO [
Using qualitative evidence in decision making for health and social interventions: an approach to assess confidence in findings from qualitative evidence syntheses (GRADE-CERQual).
for the GRADE Working Group Rating quality of evidence and strength of recommendations GRADE: an emerging consensus on rating quality of evidence and strength of recommendations.
], etc) as “theories” and they may not consider that they are using social theory in their systematic reviews.
The main difficulty in understanding the range and use of social theories available as a resource for systematic review authors is lack of common language and understanding regarding their location on the theory spectrum (Fig. 1), and the inconsistent terminology used to label and describe theories in the context of systematic review methods. Social theories are variously and inconsistently termed theories, conceptual models or frameworks, tools, taxonomies, typologies, hypotheses, propositions, conjectures, and so forth. In the context of systematic reviews, we propose “theory” as an overarching term, but also characterize two main overlapping categories: (1) theories for systematizing review processes, and (2) theories for conceptualizing, theorizing, and interpreting evidence (see Fig. 2).
Fig. 2Categorization of social theory in the context of systematic reviews. Abbreviations: PICO, Patient, Intervention, Comparison, Outcomes, Study types; PRISMA, Preferred Reporting Items for Systematic Reviews and Meta-Analyses.
Most theories located on the spectrum shown in Fig. 1 can be situated within one or other of these two categories. Some theories may however be located within either category or develop through the process of the review and move across categories or theory level as they become more fully developed and comprehensive and become more powerful in explaining phenomena; such as with the concurrent development of a logic model whilst conducting a review to systematize data processing and interpretation. For example, review authors such as Turley et al. commenced their review by developing rudimentary logic to inform the review design. This was extended within an initial logic model to identify outcomes of interest and then further refined and presented as a mid-range theory in the form of a more fully developed logic model to provide an integrated conceptual picture explaining the review findings [
1.5 Ascertaining a picture of current and potential use of theory in systematic reviews of complex interventions
The increasing trend for reviewers to incorporate social theory into their reviews presents difficult challenges related to the identification and selection of appropriate theory that might be useful and add value in specific review contexts. It is likely that the emphasis on using theory in new UK MRC guidance on designing and conducting process evaluations to understand the functioning of a complex intervention will generally lead to increased interest in theory among systematic review teams [
]. The role of theory in systematic reviews however has much greater potential than simply acknowledging the theoretical basis for interventions; theory can be deployed at every stage of a review to develop hypotheses, refine questions, select outcomes of interest, systematize processes, organize ideas, extract data, inform thinking and support interpretation of evidence, and provide a structure for reporting. Indeed, theory already underpins these stages in systematic reviews, although this contribution may not be explicitly articulated; this suggests that evidence-based medicine which determines the systematization of the standard Cochrane intervention effect review is not yet well articulated as a mid-range theory.
Methodologists within Cochrane were keen to address the apparent limitations of the standard Cochrane review approach and the lack of guidance on use of social theory in Cochrane reviews when developing new guidance on the conduct of systematic reviews in which complexity was an important consideration. Use of theory in systematic reviews was a major topic for discussion at an international meeting of global methodologists in Montebello in 2012, part-funded by the Methodological Investigation of Cochrane Complex Intervention (MICCI) reviews project grant from Cochrane. A series of published articles from the Montebello meeting articulated the potential important role of theory, particularly within complex intervention reviews, with a future research and development agenda being developed by consensus [
]. The research and development agenda outlined the need for urgent exploratory research to establish a picture of current and potential use of theory in systematic reviews. Developing a better-shared understanding of the use and value of theory is critically important as methods for conducting systematic reviews develop in response to the need to answer increasingly diverse review questions, in particular, when seeking to explain how and why complex interventions work, or do not work within any given context. New social theories, and new uses for existing theories, have proliferated to address these questions. One component of the empirical work of the MICCI project was designed to start addressing this critical evidence gap.
Our aim was to
•
Identify and present a snapshot of examples of published theories of social phenomena currently used in systematic reviews of complex interventions; with brief explanations of their potential value in systematic reviews of complex interventions, and with references to associated methodological articles and examples of reviews that had used them,
•
Develop a searchable resource of theories and reviews that used theory for review authors, and
•
Produce Cochrane Guidance on the classification, use, and selection of theory in systematic reviews of complex interventions.
2. Methods
We designed a three-stage iterative approach involving literature searching, expert engagement and consultation, and organization and classification of theories that was subsequently developed into two searchable resources for authors conducting reviews of complex interventions. We used the UK MRC definition of a complex intervention as “an intervention comprising multiple components which interact to produce change. Complexity may also relate to the difficulty of behaviours targeted by interventions, the number of organisational levels targeted, or the range of outcomes” [
]. Data collection and analysis was carried out between January 2013 and September 2014. We then developed Cochrane Guidance for review authors on the use of theory in systematic reviews of complex interventions.
Fig. 3Typology of complexity in complex interventions
2.1 Stage 1 searching for published and unpublished examples of theories and creating an initial database
We set out to identify examples of published and unpublished systematic reviews of complex interventions that incorporated social theories, with brief explanations of the potential added value of the theory in systematic reviews of complex interventions, and with references to associated methodological articles and further examples of reviews that had used them. To have the most contemporary picture, we also sought to identify new theories designed for or that could have potential application in systematic reviews, irrespective of whether it had yet been used in a systematic review. We therefore included reference to unpublished systematic reviews that used theories of particular novelty of interest. At the outset, we were aware that the rate of development of new approaches to systematic reviewing is too rapid, and the proliferation of theories and ways in which they are applied in systematic reviews too great, to allow us to name, let alone describe all of them. We therefore aimed to identify and present a selective snapshot of examples to raise awareness of theories and provide Cochrane Guidance to encourage review authors to think about when it is appropriate to use theory in their review and the potential added value that this might bring. Although many reviews (especially qualitative evidence syntheses such as metaethnography) are designed to develop new theory, in the context of this methodological work, we primarily focused on where social theories have been used to enhance the conduct of a systematic reviews and the interpretation of evidence.
An iterative, consultative approach was adopted by the research team for the following reasons:
1.
Results from scoping searches in Google and Google Scholar proved overwhelming. Terms such as “theory,” “model,” and “framework” occur very frequently in the context of the health and social care systematic reviews; an exhaustive list of other terms, that is, the plethora of names of recently developed tools, could not be generated comprehensively. Therefore, we could not reliably construct a search strategy with sufficient sensitivity and specificity for use in either bibliographic databases or Internet search engines.
2.
Theory development is a rapidly expanding field; we knew from personal contacts in the global systematic review methodology community that a number of tools were currently in developmental or in prepublication stages.
Expert consultation was used as the main approach to identifying a snapshot of the current use of social theory in complex intervention reviews. In January 2013, we circulated a request to MICCI project coapplicants and collaborators (n = 30), Montebello meeting attendees (n = 50) and an e-mail list, managed by Cochrane, of global systematic reviewers with an interest in developing methods for conducting reviews of complex interventions in health and social care (n = 70). There was some overlap between lists; when duplicates were accounted for this group consisted of around 100 people who were generally key methodologists and highly experienced systematic reviewers known to undertake Cochrane and non-Cochrane reviews of different types and designs. Initially, we asked to be informed of any review protocols, review reports, or articles that incorporated a theory as defined in Fig. 2, or any methodological articles that described or evaluated methods for using theory in any part of a systematic review.
We collated the information received by recording the name and/or a brief description of each theory, the theoretical background on which it was based, examples (if any) of systematic reviews using any design in which it had been used, and authors' comments about its usefulness or potential usefulness in reviews of complex interventions. Many people responded to this request, others forwarded it to colleagues with one contact often leading to another via snowball sampling. Other theories were identified by searching the bibliographies of papers, from our initial scoping searches of Google and Google scholar, or serendipitously in the course of other reading. Where necessary, we asked authors for further clarification as to whether any additional methodological work had been undertaken, and whether the theory had been used (or was being used) in a systematic review. The purpose was to be illustrative rather than exhaustive.
Response to our e-mail requests was surprisingly high, yielding information on a large number of theories. Some theories were already known to us, but many were new and recently developed. To decide how to handle this large volume of material, we convened an open workshop at the 2012 Cochrane Colloquium in Auckland for feedback and comment from 30 collaborators and key methodologists many who had attended a meeting in Montebello, Canada in January 2012 on commencement of this work. Following feedback, it was agreed to categorize the theories according to their use in the systematic review process (as shown in Fig. 1, Fig. 2) and focus most attention on obtaining further information on theories that could potentially aid understanding of intervention complexity, many of which were newer and less well known.
2.2 Stage 2 categorizing, organizing, and clarifying theories
We reviewed each database entry and then created two further databases. The first included mainly low-level theories concerned with systematizing review processes and commonly in use. The second, and potentially more important, database contained theories that could potentially be helpful in designing, conducting, and interpreting the findings of complex intervention reviews. We noted any theory for which full details were not either published or made available to us by the authors. We contacted the authors again with theory-specific questions such as:
•
Has the [name of theory] you developed been used in a systematic review?
•
If so, can we cite this review as an exemplar?
•
Has the [name of theory] undergone any further development or evaluation?
We also requested authors' comments on key points to be included in guidance for any reviewers who were considering using their “theory” and annotated each relevant entry.
2.3 Stage 3 developing resources and guidance for review authors
One of the authors (A.B.) developed a searchable Wiki and a Methodology Register in Mendeley as a review author resource by using data and references from stages 1 and 2 with the intention that it would be augmented over time. Finally, using evidence from stages 1 and 2, we developed Cochrane Guidance for review authors on how to identify, choose, and use theory in systematic reviews of complex interventions to supplement the two searchable resources.
3. Results
Over the last 10 years, with a notable proliferation within the last 5 years, authors have incorporated social theory in every stage of a systematic review from the design and protocol stage to the interpretation of findings. Review authors have used social theory both to standardize and to innovate systematic review methods. We collated details of over 100 theories and briefly described them in tabular form, organized by the stage of review in which they might be deployed. For illustrative purposes, Table 1 summarizes a selection of commonly used theories to systematize the review process and Table 2 summarizes selected examples of theories that could be used to enhance review design and data processing and interpretation in systematic reviews of complex interventions.
Table 1Some selected examples of low-level theories for systematizing review processes
Review process
Example of theory
Planning the review and formulating the review question
PICOS (Patient, Intervention, Comparison, Outcomes, Study types) and alternative frameworks for different review types help in planning the review and framing the review question. Organizations such as the Cochrane (www.Cochrane.org/) and the EppiCentre (eppi.ioe.ac.uk/) offer a framework and software for conducting a review that is compatible with their specific “brand.”
Searching the literature
Search strategies are tailored to an individual review question, but methods for documenting the search processes can be standardized, and search results should be reported in a PRISMA flow diagram, available from http://www.prisma-statement.org/.
Data collection
PICOS (or alternative) informs inclusion/exclusion criteria and aids study selection Data-extraction forms are often designed to suit individual reviews but may be based on standardized templates, for example, the example provided by the Centre for Research and Dissemination at York University available from http://www.york.ac.uk/inst/crd/SysRev/!SSL!/WebHelp/1_3_UNDERTAKING_THE_REVIEW.htm. The National Institute for Health and Care Excellence (NICE) provides a standard tool for its reviews (Methods for the Development of NICE Public Health Guidance. 2nd edn. National Institute for Health and Clinical Excellence, London, 2009. Appendix K)
Quality appraisal/assessment of risk of bias
The Critical Appraisal Skills Programme (CASP) provides a range of tools for appraising the quality of individual studies with different designs, available from http://www.casp-uk.net/#!casp-tools-checklists/c18f8. The GRADE working group provides a framework and software for grading the quality of evidence and the strength of recommendations available from http://www.gradeworkinggroup.org/index.htm.
Synthesizing the evidence
The Cochrane Handbook provides a general framework for synthesis, whether quantitative or narrative, in chapter 9, available from www.cochrane-handbook.org.
Reporting the findings
The PRISMA statement with checklist and flow diagram available from http://www.prisma-statement.org/ is intended to standardize good practice in reporting systematic reviews.
Abbreviation: PRISMA, Preferred Reporting Items for Systematic Reviews and Meta-Analyses.
Table 2Some examples of theories that have been deployed in systematic reviews of complex interventions
Theory
Theoretical background
Use in reviews of complex interventions
Example systematic review
Behavior change taxonomies (BCTs); low-level midrange theory
The first cross-behavior classification system to demonstrate interrater reliability in identifying 22 BCTs and four BCT packages in descriptions of interventions was published in 2008. Building on this and five other taxonomies, Michie et al. developed BCT Taxonomy v1; the first cross-behavior, hierarchically organized taxonomy, established by international expert consensus and comprising 93 clearly labeled, well-defined behavior change techniques with demonstrated reliability in specifying 26 of the most frequently occurring BCTs: Michie, S., Abraham, C., Eccles, MP., et al. (2011). Strengthening evaluation and implementation by specifying components of behavior change interventions: a study protocol. Implement Sci., 6. Michie, S., Richardson, M., Johnston, M. et al. (2013). The Behavior Change Technique Taxonomy (v1) of 93 Hierarchically Clustered Techniques: Building an International Consensus for the Reporting of Behavior Change Interventions, Ann. Behav. Med. 46, 81–95.
In systematic reviews of complex interventions, this approach allows the specification of intervention content into its component behavior change techniques. By combining this with the statistical technique of metaregression and theory-driven analyses, commonly occurring BCTs associated with effective outcomes can be identified. BCTs have been used by NICE in the systematic reviews for its 2012/13 update of its Behaviour Change Guidance (http://www.nice.org.uk/nicemedia/live/13596/59328/59328.pdf). A Web-based users' resource is available, including the most recent version of the taxonomy, guidance on its use, and a discussion board for questions, comments, and feedback. www.ucl.ac.uk/health-psychology/BCTtaxonomy/ There is an online training course for using behavior change techniques in specifying complex interventions. http://www.ucl.ac.uk/health-psychology/bcttaxonomy/Online_training
National Institute for Health and Clinical Excellence (2007). Health systems and health-related behaviour change: a review of primary and secondary evidence. London: National Institute for Health and Care Excellence Michie, S., Jochelson, K., Markham, WA., & Bridle, C. (2009). Low-income groups and behaviour change interventions: a review of intervention content, effectiveness and theoretical frameworks. J Epidemiol. Comm. Health, 63. 610–622. Dombrowski, SU., Sniehotta, FF., Avenell, A., Johnston, M. et al. (2012). Identifying active ingredients in complex behavioural interventions for obese adults with obesity-related co-morbidities or additional risk factors for co-morbidities: a systematic review. Health Psychology Review. 6(1). 7–32. Bird, EL., Baker, G., Mutrie, N., Ogilvie, D., Sahlqvist, S., Powell, J. (2013). Behavior Change Techniques Used to Promote Walking and Cycling: A Systematic Review. Health Psychology.
May, C., Murray, E., Finch, T., Mair, F., Treweek, S., Ballini, L., Macfarlane, A. and Rapley, T. (2010) Normalization Process Theory On-line Users' Manual and Toolkit. Available from http://www.normalizationprocess.org [Accessed on 16th January 2015].
NPT can provide a valuable method to aid the conduct and interpretation of systematic reviews of a range of different types of qualitative study and that there are three main ways in which it could be used: To support the development of research questions and overall design of a systematic review. To serve as a framework for data analysis within a systematic review. To support the interpretation of a systematic review's results.
Mair F, May C, Murray E, Finch T, O'Donnell C, Anderson G, Wallace P, Sullivan F. Understanding the implementation and integration of e-Health Services. Report for the NHS Service and Delivery Organisation R&D (NCCSDO). 2009. London. SDO. www.sdo.nihr.ac.uk May C, Finch TL, Cornford3 J, Exley C, Gately4 C, Kirk5 S, Jenkings6 KN, Osbourne7 J, Robinson2 AL, Rogers A, Wilson R, Mair FS. Integrating telecare for chronic disease management in the community: what needs to be done? Department of Health 2010, London.
Frameworks for evidence synthesis based on psychological theories; midrange theories
Glanz K, Bishop DB: The role of behavioral science theory in development and implementation of public health interventions. Annu Rev Public Health 2010, 31:399–418. Painter JE, Borba CPC, Hynes M, Mays D, Glanz K: The use of theory in health behavior research from 2000 to 2005: a systematic review. Ann Behav Med 2008, 35:358–362. Filiatrault J, Richard L: Theories of behavior change through preventive and health promotion interventions in occupational therapy. Can J Occup Ther 2005, 72:45–56.
Psychological theories can provide a useful basis from which to develop a framework for data analysis and synthesis. In the case of the example reviews, the Health Belief Model was chosen because it was used in several of the included studies and thus offered a useful starting point for developing codes to analyze the findings. National Cancer Institute. Theory at a Glance: A Guide for Health Promotion Practice, 2nd edn. US Department of Health and Human Sciences, Bethesda, MD: National Institutes of Health, 2005.
Garside R, Pearson M, Moxham T. What influences the uptake of information to prevent skin cancer? A systematic review and synthesis of qualitative research. Health Education Research 2009; 25:1 162–182. Lorenc T, Jamal F, Cooper C. Resource provision and environmental change for the prevention of skin cancer: systematic review of qualitative evidence from high-income countries. Health Promotion International; 2012 http://dx.doi.org/10.1093/heapro/das015
Using logic models in a systematic review; midrange theories
Anderson LM, Petticrew M, Rehfuess E, Armstrong R, Ueffing E, Baker P, Francis D, Tugwell P. Using logic models to capture complexity in systematic review. Research synthesis methods 2011, 2:33–42 Turley R, Saith R, Bhan N, Doyle J, Jones K, Waters E. Slum upgrading review: methodological challenges that arise in systematic reviews of complex interventions. Journal of public health 2013; 35:1, 171–175 Tugwell P, Petticrew M, Kristjansson E, Welch V, Ueffing E, Waters E, et al. Assessing equity in systematic reviews: realizing the recommendations of the Commission on Social Determinants of Health. BMJ 2010; 341:c4739 Kellogg Foundation. Logic model development guide. www.wkkf.org/knowledge-center/resources/2006/02/ WK-Kellogg-Foundation-Logic-Model-Development-Guide.aspx (accessed 7 February 2012) Harris RP, Helfand M, Woolf SH, Lohr KN, Mulrow CD, Teutsch SM, et al. Current Methods of the U.S. Preventive Services Task Force: A Review of the Process. American journal of preventive medicine. 2001; 20 (35).
Logic models can be used at different stages, for example, scoping the review, refining and conducting the review, making the review relevant to policy and practice. Turley et al. developed a logic model at the protocol stage of their review to describe potential components of slum upgrading strategies, whereas Glenton et al. developed their logic model to integrate their qualitative findings about interventions delivered by lay health workers with the results of a separately conducted effectiveness review. In review on preschool feeding, a logic model was developed to make assumptions about the program explicit, and the assumptions were tested in the synthesis and analysis.
Turley R, Saith R, Bhan N, Rehfuess E, Carter B. Slum upgrading strategies involving physical environment and infrastructure interventions and their effects on health and socio-economic outcomes. Cochrane database of systematic reviews 2013, Issue 1. http://dx.doi.org/10.1002/14651858.CD010067.pub2. Glenton C, Colvin CJ, Carlsen B, Swartz A, Lewin S, Noyes J, Rashidian A. Barriers and facilitators to the implementation of lay health worker programmes to improve access to maternal and child health: qualitative evidence synthesis. Cochrane Database of Systematic Reviews; 2013, Issue 10. http://dx.doi.org/10.1002/14651858.CD010414.pub2. Kristjansson E, Francis DK, Liberato S, Benkhalti Jandu M, Welch V, Batal M, Greenhalgh T, Rader T, Noonan E, Shea B, Janzen L, Wells GA, Petticrew M. Feeding interventions for improving the physical and psychosocial health of disadvantaged children aged three months to five years. Cochrane database of systematic reviews 2012, Issue 6. http://dx.doi.org/10.10002/14651858.CD009924.
Dealing with diverse interventions: developing and prioritizing outcome categories; low-level theory
Because there are multiple approaches to problems, the authors of these example reviews devised a conceptual framework for the intervention and developed “outcome categories.”
Key points: 1. Determine a priori the process to use to categorize outcomes in included studies, including how you will choose an outcome when more than one is included in an outcome category 2. Think about how you would select a time point for outcomes measured at multiple time points 3. Think about what you will do if the same outcome is measured in different studies but is not selected through the process you determine in point 1 above (for example, if you have a category called “treatment outcomes” and 3 of 4 studies measure a similar outcome, such as cholesterol level, but it does not meet the selection criteria you have established for choosing a treatment outcome in some studies will you report it as an additional outcome?)
Horvat L, Horey D, Romios P, Kis-Rigo J. Cultural competence education for health professionals. Cochrane Database of Systematic Reviews 2014, Issue 5. Art. No.: CD009405. http://dx.doi.org/10.1002/14651858.CD009405.pub2. Horey D, Kealy M, Davey MA, Small R, Crowther CA. Interventions for supporting pregnant women's decision-making about mode of birth after a caesarean. Cochrane Database of Systematic Reviews 2013, Issue 7. Art. No.: CD010041. http://dx.doi.org/10.1002/14651858.CD010041.pub2
Numerous low-level theories have been designed for the purpose of systematizing review processes. Many were well-known, some to the extent that they have become more or less absorbed into standard practice for systematic reviews of effectiveness, pharmacological interventions, or diagnostic test accuracy (Table 1). For example, every Cochrane review is expected to begin with a theory of how the intervention is intended to work, and the whole Cochrane template, embedded in RevMan software (Version 5.3, The Nordic Cochrane Centre, Copenhagen, as well as its component parts such as PICO) could be described as an overarching framework within which to systematize the review conduct and reporting. We chose not to include many such examples in the database as this represents the norm.
The proliferation of development of new low-level theories to systematize review processes now extends beyond the effectiveness review to include other review types and designs with particular relevance for complex intervention reviews. For example, since 2000, GRADE has been developed to determine the confidence in findings for effect reviews [
for the GRADE Working Group Rating quality of evidence and strength of recommendations GRADE: an emerging consensus on rating quality of evidence and strength of recommendations.
Barriers and facilitators to the implementation of lay health worker programmes to improve access to maternal and child health: qualitative evidence synthesis.
Using qualitative evidence in decision making for health and social interventions: an approach to assess confidence in findings from qualitative evidence syntheses (GRADE-CERQual).
]. Three tools to systematize review processes developed in response to specific gaps identified in the research and development agenda, published following the 2012 meeting of methodologists in Montebello, are yet to be fully tested; the TIDieR tool for reporting complex interventions [
Not surprisingly, mid-range theories, commonly used in primary studies, are often transferred without adaptation for use in systematic reviews to inform the review design and data interpretation. Reviews that used mid-range theory were more commonly conducted outside of a Cochrane context. For example, Garside et al. used the Health Belief Model as the conceptual framework to extract and interpret evidence in their qualitative evidence synthesis of influences on the uptake of information to prevent skin cancer [
]. Normalization Process Theory developed by May et al. has also gained some traction as a framework of choice for conceptualizing implementation in complex intervention reviews [
]. If a bespoke theory is not available, Booth et al. have developed an approach whereby if the theory is a reasonable, but not optimal, fit for the review, then it can be adapted to facilitate a “Best Fit” Framework Synthesis; there are several examples of this approach used in a review [
]. Conversely, we also noted theories used thus far solely by their originators (see for example, the “effectiveness plus” model developed by Snilsveit [
], increasing examples of this particular use of mid-range theory have been reported. In their mixed-method systematic review protocol, Hurley et al. developed two conceptual logic diagrams from an initial synthesis of literature to show the effects of erroneous health beliefs and the complex reciprocal interrelationship between pain, physical, and psychosocial function and exercise interventions [
]. Turley et al. developed an a priori logic model that was developed over the course of the review exploring the effectiveness of slum upgrading initiatives [
]. Glenton et al. used a logic model as a means of integrating a qualitative evidence synthesis on implementation with the findings of a Cochrane effectiveness review on community health workers [
Barriers and facilitators to the implementation of lay health worker programmes to improve access to maternal and child health: qualitative evidence synthesis.
Psychologists such as Michie have had considerable influence on methodological development of low-level and mid-range theory for the conduct of systematic reviews of behavior change interventions (for example, taxonomies of behavior change interventions and a behavior change wheel) [
The behavior change technique taxonomy (v1) of 93 hierarchically clustered techniques: building an international consensus for the reporting of behavior change interventions.
We also were notified of an updated review where the authors had taken the opportunity to reassess their methods and introduce a theory when updating. The 2012 Cochrane review of audit and feedback effects on professional practice and health outcomes updated an earlier version that did not draw on theory and resulted in no clear pattern of findings. The updated version of the review reanalyzed the data using the mid-range Control Theory finding support for the hypothesis that adding goals or targets and action plans to feedback interventions improved effectiveness. This proved a useful finding given that very few audit and feedback interventions included these components [
We were unable to identify any reviews in the field of health and social care that incorporated an explicit grand theory (beyond being located in evidence-based medicine or health care contexts), neither through targeted literature searching, due to the lack of specificity in currently available search techniques, nor via the consultation process. It is likely that such reviews do exist in a health and social care context even if the theory is not explicitly stated. Use of grand theory such as Feminist theory is common in primary research in a health and social care context, and published examples exist in reviews in advertising, media and business [
Navarro-Beltrá M, Llaguno MM: A systematic review of gender and advertising studies. Catalan J Commun Cult Stud 4(2 – Special Issue: Gender Relations in the Media: Conflicting Pictures and New Representational Practices). http://dx.doi.org/10.1386/cjcs.4.2.171_1.
3.4 Added value of using theory in a systematic review of complex interventions
Convention dictates that there are core set of low-level theories in the form of systematic review tools and reporting standards that add value in systematizing review processes. Although these may not be commonly thought of as theories within the standard Cochrane intervention effect review template, they reveal an underlying set of understandings from an evidence-based medicine perspective about how impact comes about and how it should be measured [
for the GRADE Working Group Rating quality of evidence and strength of recommendations GRADE: an emerging consensus on rating quality of evidence and strength of recommendations.
] reporting standards, and so forth have become mandatory. Beyond this core set of low-level theories, a large number of tools exist from which review authors are able to select. However, few published reports or evaluations exist to establish, beyond the testimony of their originators, the added value of incorporating low, mid-range, and grand theory into systematic reviews. Unless authors publish their experiences of using particular theories, and the difference (or not) they made, it is problematic to determine their usefulness.
Reviewers who used mid-range theoretical frameworks in their reviews said that such theories enabled a greater depth of inquiry and more nuanced interpretations of findings. More instrumental use of conceptual frameworks is believed to facilitate the speed and efficiency of data extraction [
]. Review authors report that expertise and team development is needed to fully engage with the specific theory. In a published report, Turley et al. outlined their experiences of developing and using a logic model. They identify the additional advantages, as well as the challenges, that the review team encountered in what appeared to be a long and convoluted process [
]. In contrast, authors of a qualitative evidence synthesis, report being overly constrained by an a priori theoretical framework and having to change tack mid review. Thomas and Harden developed an inductive line by line approach to thematic synthesis having previously given up on trying to develop an initial a priori framework to explain children's conceptualizations as to why they do and do not eat fruit and or vegetables [
]. Overall, we do not have a clear picture of when and how review teams select mid-range theories in the review process, or how common it is for theories to be tried, modified, or discarded if they do not add value. Nor is it clear how to kit together use of theories in a sensible and coherent way, and there is little documented experience of the optimal number of theories in any given review.
3.5 Theory in Reviews Wiki and Mendelay Theory Inventory
The “Theory in Reviews” Wiki http://theoryinreviews.pbworks.com/will be maintained as part of the study register activities of the Cochrane Qualitative and Implementation Methods Group. It includes examples of theories that can be used in the systematic review process, particularly in systematic reviews of complex interventions. Theories are listed under each stage of the review process. Links to full text records, or to abstracts where full text is not openly available, are given within the individual wiki pages. The wiki is searchable, using an internal search engine you can identify theories by, for example author (e.g., May author of Normalisation Process Theory) or theory name (e.g., Behemoth).
Articles identified during the search are also tagged for social bookmarking via the Mendeley Theory in Reviews Inventory as a free searchable resource for authors to find and locate studies and reviews that report or use theory that may be of interest to review author. The inventory will also be updated periodically (http://www.mendeley.com/groups/4714181/).
Authors and methodologists are invited to notify the convenors of the Cochrane Qualitative and Implementation Methods Group of any new or additional publications via their web site (http://cqim.cochrane.org/).
3.6 First available guidance for review authors on the classification, choice, and use of theory in complex intervention reviews
There can be many intervention strategies in complex interventions (i.e., things that the researches “do” and/or provide to participants). It is therefore possible that more than one theory may be needed to explain the rationale behind each intervention strategy and/or explain how and why it produces an outcome. The Cochrane guidance for review authors (see supplemental online file and citation Noyes et al [
]) provides a framework (i.e., low-level theory) for the identification, selection, and use of theory in complex intervention reviews with reference to the searchable Wiki and Mendeley Inventory. Criteria of “good” and “bad” theory are outlined (Fig. 4). The guidance also recommends use of BeHEMoTh (Behavior of Interest—Health Condition or Setting—Exclusions—Models or Theories) as a tool for searching for theories [
This article reports the first snapshot of the use of social theory in systematic reviews addressing complex health and social care questions and provides new insights into the range and extent of theory used. Given that widely used systematic review methods guidance such as the Cochrane Handbook [
Higgins J.P.T. Green S. Cochrane handbook for systematic reviews of interventions Version 5.1.0 [updated March 2011]. The Cochrane Collaboration,
London, UK2011
] barely mention the use of social theory, apart from low-level theory in the form of tools to systematize review processes, it was particularly surprising to document how prevalent use of social theories, especially midrange theories, has been in published systematic reviews of complex health and social care interventions. It is however important to acknowledge that there is mathematical theory in systematic review methods such as network meta-analysis (statistical and geometric theory), and different levels of social theory underpinning the overarching context of evidence-based medicine and health care that is not made explicit in systematic review manuals. It appears that the increasing number of qualitative researchers from a sociological tradition who now undertake theory-informed systematic reviews may have influenced the introduction of familiar social theories used in primary qualitative research into complex intervention systematic review methods and processes. For example, Popay et al.'s Narrative Synthesis Guidance published in 2006 was strongly influenced by sociologists and outlined a four-stage approach starting off with developing a social theory of how the intervention or implementation worked [
]; the examples shown are midrange logic models. Similarly, most complex interventions involve behavior change and key methodologists and researchers from a psychology tradition have developed new theories that have been adopted in complex intervention reviews. The most recent MRC guidance on the design of process evaluations for complex interventions recommends development of a midrange logic model and consideration of the use of midrange complexity theory to guide analysis and interpretation [
]. Newer theory-informed review approaches such as realist and metanarrative reviews are also increasing the visibility and potential of using and developing theory as part of the systematic review process. Most recently, methods for undertaking reviews of theory have been published, which give further prominence to the potential use of theory in systematic reviews [
Although novel and the first methodological work of this type in the context of systematic reviews, this work does have some limitations. It was not possible to conduct a systematic search for examples of the use of social theory in systematic reviews of health and social care interventions, and thus, the aim was to present illustrative examples and not to be exhaustive. Nor do the examples provided cover the full range of theories that may be appropriate for specific review contexts. However, there will be an opportunity to add further examples to the “Theory in Reviews” Wiki and Mendeley Inventory over time. Although this study was funded by Cochrane, a strength is that the expert methodologists and reviewers consulted represent a wide range of influential systematic review interests and were not confined to Cochrane. Although use of snowballing techniques widened the reach to other reviewers and methodologists, we cannot establish how representative those consulted are of the entire methods and complex intervention systematic review community.
5. Conclusion
Social theory, especially low and midrange theory, is increasingly used throughout every stage and process in systematic reviews and especially in complex intervention reviews. Choice of theory remains a personal preference and is constrained by the knowledge and disciplinary backgrounds of the review team. Effective application of theory in the future is likely to depend on such factors as the review question, suitability of the theory, the type and quality of the data, the skills of the review team and the time available to complete the review. Further methodological research is needed to unpack and evaluate the use and added value of theory in systematic reviews, particularly in relation to the systematic identification and quality assessment of candidate theories. Where theories are used to explain phenomena, review teams need to decide which explanation is closer to the “truth.” For any one observed phenomenon, there are however often multiple possible explanations. How to decide between them requires specific attention and further research.
Acknowledgments
Authors' contributions: J.N. conceived the original idea and outline plan for the research, which was coordinated by J.C. on behalf of Cochrane for the purposes of developing a chapter on complex interventions for the Cochrane Handbook. All authors contributed to further development of the methods. M.H. collected data and conducted the analysis and, in collaboration with J.N., the interpretation of the data. C.G. and S.L. contributed to study design and data interpretation. A.B. conceived and developed the study registers. R.G. shared additional references to support development of the guidance. All authors read and critically revised drafts of the article.
The authors also acknowledge the valuable contribution of a large number of people including: wider MICCI project coapplicants and collaborators who attended a meeting in Montebello in January 2012 and those who supported the project throughout; participants who attended the meetings and workshops at the Cochrane Colloquium in Auckland, New Zealand, in October 2012; MICCI workshop attendees at the Cochrane midyear meeting in Oxford in March 2013, those who attended an MICCI meeting and the Methods Symposium at the 2013 Cochrane Colloquium in Quebec City, and attendees at the Methods symposium at the 2014 Cochrane Colloquium in Hyderabad in September 2014; and all others who contributed by e-mail to the development of the databases. There are too many people to name individually and the authors thank them all for their support with this important work. The authors thank the peer reviewers for their excellent feedback. The views expressed are those of the authors and not necessarily those of Cochrane or its registered entities, committees, or working groups.
Process evaluation of complex interventions: Medical Research Council guidance.
in: MRC Population Health Science Research Network. 2014 (London. Available at: https://www.mrc.ac.uk/documents/pdf/mrc-phsrn-process-evaluation-summary-guidance/. Accessed January 25, 2016)
Using the internet to promote health behaviour change: a systematic review and meta-analysis of the impact of theoretical basis, use of behaviour change techniques and mode of delivery on efficacy.
Barriers and facilitators to the implementation of lay health worker programmes to improve access to maternal and child health: qualitative evidence synthesis.
Higgins J.P.T. Green S. Cochrane handbook for systematic reviews of interventions Version 5.1.0 [updated March 2011]. The Cochrane Collaboration,
London, UK2011 (Available at:)
Using qualitative evidence in decision making for health and social interventions: an approach to assess confidence in findings from qualitative evidence syntheses (GRADE-CERQual).
The behavior change technique taxonomy (v1) of 93 hierarchically clustered techniques: building an international consensus for the reporting of behavior change interventions.
Navarro-Beltrá M, Llaguno MM: A systematic review of gender and advertising studies. Catalan J Commun Cult Stud 4(2 – Special Issue: Gender Relations in the Media: Conflicting Pictures and New Representational Practices). http://dx.doi.org/10.1386/cjcs.4.2.171_1.
Conflicts of interest: J.N., A.B., and R.G. are coconvenors of the Cochrane Qualitative and Implementation Methods Group. J.N. is a co-chair of the Cochrane Methods Executive and member of the Methods Application and Review Standards group. S.L. and C.G. are both editors for the Cochrane Effective Practice and Organisation of Care Group and the Cochrane Consumers and Communication Review Group. J.C. is Cochrane Methods co-ordinator.
Funding: The authors gratefully acknowledge Cochrane for funding the MICCI project from the Methods Innovation Fund. Bangor University contributed 40% of Noyes' time for 3 years. Cochrane funded Chandler's time. The University of Sheffield contributed 2% of Booth's time over the equivalent of a 1-year period for database development and for information management advice.
00005-6&id=gr1.jpg){kind=link}
00005-6&id=gr2.jpg){kind=link}
00005-6&id=gr3.jpg){kind=link}
00005-6&id=gr4.jpg){kind=link}