Advertisement

Living systematic review: 1. Introduction—the why, what, when, and how

Published:September 10, 2017DOI:https://doi.org/10.1016/j.jclinepi.2017.08.010

      Abstract

      Systematic reviews are difficult to keep up to date, but failure to do so leads to a decay in review currency, accuracy, and utility. We are developing a novel approach to systematic review updating termed “Living systematic review” (LSR): systematic reviews that are continually updated, incorporating relevant new evidence as it becomes available. LSRs may be particularly important in fields where research evidence is emerging rapidly, current evidence is uncertain, and new research may change policy or practice decisions. We hypothesize that a continual approach to updating will achieve greater currency and validity, and increase the benefits to end users, with feasible resource requirements over time.

      Keywords

      What is new?
      • Living systematic review was proposed in 2014 as an approach to systematic review updating. Since that time, many issues associated with the production and publication of living systematic reviews have been explored and are now being tested in a series of living systematic review projects (Cochrane and non-Cochrane).
      • This series presents the current state of development of living systematic reviews by members of the international Living Systematic Review Network.
      • This paper gives an overview of recent living systematic review developments in production and publication, including a working definition of living systematic review, when living systematic reviews are appropriate, and current approaches to living systematic review production.
      This paper is the first in a series published in this issue of the journal, providing an overview of living systematic reviews (LSRs) and living guideline recommendations. This introductory paper introduces the why, what, when, and how of LSRs. Key issues in LSRs are discussed, including searching, updating scenarios, production processes, editorial and peer review, and publication. Other papers in the series explore the contribution from new technologies, such as text mining, machine learning, and crowd sourcing (Thomas et al., this issue); examine the statistical issues associated with repeated meta-analysis (Simmonds et al., this issue); and describe the opportunities to link LSRs with living guidelines (Akl et al., this issue).

      1. Keeping evidence up to date

      Health research is a key driver of health and well-being, but health professionals often make decisions based on a patchy understanding of what the research says; consumers are confronted by disparate and often conflicting research findings; and society's return on investment in health research is eroded as research findings are lost in the deluge of new research [
      • Macleod M.
      • Michie S.
      • Roberts I.
      • Dirnagl U.
      • Chalmers I.
      • Ioannidis J.
      • et al.
      Biomedical research: increasing value, reducing waste.
      ]. Over the last 30 years, Cochrane and others have tackled these challenges by developing the science of evidence synthesis, designed to enable health decisions informed by the best available evidence.
      Achieving the goal of providing the “best available evidence” requires careful attention to the methods used to identify, appraise, and synthesize relevant research. These methods are well developed [
      • Higgins J.
      • Lasserson T.
      • Chandler J.
      • Tovey D.
      • Churchill R.
      Methodological expectations of cochrane intervention reviews.
      ,
      • Moher D.
      • Liberati A.
      • Tetzlaff J.
      • Altman D.
      The PRISMA Group. Preferred reporting Items for systematic reviews and meta-analyses: the PRISMA statement.
      ,
      Institute of Medicine
      ], but resource intensive [
      • Tricco A.C.
      • Brehaut J.
      • Chen M.H.
      • Moher D.
      Following 411 Cochrane protocols to completion: a retrospective cohort study.
      ], making it difficult to keep up to date as new research becomes available [
      • Bastian H.
      • Glasziou P.
      • Chalmers I.
      Seventy-five trials and eleven systematic reviews a day: how will we ever keep up?.
      ]. Systematic reviews that are out of date and have not incorporated recent data are at risk of serious inaccuracy [
      • Shojania K.G.
      • Sampson M.
      • Ansari M.T.
      • Ji J.
      • Doucette S.
      • Moher D.
      How quickly do systematic reviews go out of date? A survival analysis.
      ]. Indeed, systematic review currency is challenged by the increasing rate of research output [
      • Bastian H.
      • Glasziou P.
      • Chalmers I.
      Seventy-five trials and eleven systematic reviews a day: how will we ever keep up?.
      ], which increases the frequency with which reviews need to be updated to remain accurate, and increasing research volume and rising methodological expectations which increase the work of updating.
      Conventionally, systematic reviews are not updated or updated intermittently [
      • Garner P.
      • Hopewell S.
      • Chandler J.
      • MacLehose H.
      • Akl E.
      • Beyene J.
      • et al.
      When and how to update systematic reviews: consensus and checklist.
      ]. Intermittent updating leaves gaps between updates during which time the systematic review may be missing important new research, placing it at risk of inaccuracy [
      • Shojania K.G.
      • Sampson M.
      • Ansari M.T.
      • Ji J.
      • Doucette S.
      • Moher D.
      How quickly do systematic reviews go out of date? A survival analysis.
      ] and wasting the potential contribution of new research to evidence synthesis and decision-making [
      • Créquit P.
      • Trinquart L.
      • Yavchitz A.
      • Ravaud P.
      Wasted research when systematic reviews fail to provide a complete and up-to-date evidence synthesis: the example of lung cancer.
      ]. Furthermore, reassembling an author team and restarting the review process is often difficult, and much of the institutional memory of the original team can be lost. So much work needs to be repeated that many teams describe updating as “just like starting a review from scratch.”

      2. Continual updating

      We have proposed a continual approach to review updating termed “living systematic review” (see Box 1) [
      • Elliott J.H.
      • Turner T.
      • Clavisi O.
      • Thomas J.
      • Higgins J.P.
      • Mavergames C.
      • et al.
      Living systematic reviews: an emerging opportunity to narrow the evidence-practice gap.
      ]. We hypothesize that this approach will achieve greater currency, and therefore accuracy and benefits to end users, with feasible resource requirements over time.
      Living systematic reviews
      • A systematic review that is continually updated, incorporating relevant new evidence as it becomes available
      • An approach to review updating, not a formal review methodology
      • Can be applied to any type of review
      • Uses standard systematic review methods
      • Explicit and a priori commitment to a predetermined frequency of search and review updating
      Overview of living systematic review series
      • Living systematic reviews: 1. Introduction—the why, what, when, and how
        • Authors: Julian Elliott, Annie Synnot, Tari Turner, Mark Simmonds, Eli Akl, Steve McDonald, Georgia Salanti, Joerg Meerpohl, Harriet MacLehose, John Hilton, David Tovey, Ian Shemilt, James Thomas on behalf of the Living Systematic Review Network
        • Description: Julian Elliott et al. provide an introduction and overview of the why, what, when, and how of living systematic reviews. Key issues in living systematic review are discussed, including search, updating scenarios, production processes, editorial and peer review, and publication and links to downstream activities, including living recommendations.
      • Living systematic reviews: 2. Combining human and machine effort
        • Authors: James Thomas, Anna Noel-Storr, Byron Wallace, Steven McDonald, Chris Mavergames, Paul Glasziou, Iain Marshall, Ian Shemilt, Annie Synnot, Tari Turner, Julian Elliott on behalf of the Living Systematic Review Network
        • Description: James Thomas et al. outline how new technologies which utilize human and machine effort, such as text mining, machine learning, and crowd sourcing, can enable significant production efficiencies, increasing the feasibility of continual updating. They discuss the potential and limitations of these systems, where they are in use, and where further research is needed.
      • Living systematic reviews: 3. Statistical methods for updating meta-analyses
        • Authors: Mark Simmonds, Georgia Salanti, Phillipe Ravaud, Julian Higgins, Jo McKenzie, Julian Elliott on behalf of the Living Systematic Review Network
        • Description: Living systematic reviews often require repeated meta-analyses, which may increase the risk of spurious statistical significance. Mark Simmonds et al. discuss important issues relevant to meta-analyses and network meta-analyses within living systematic reviews. Various methods that have been proposed to address these issues are reviewed and described.
      • Living systematic reviews: 4. Living guideline recommendations
        • Authors: Elie Akl, Joerg Meerpohl, Julian Elliott, Lara Kahale, Holger Schunemann on behalf of the Living Systematic Review Network
        • Description: “Living guidelines” hold the promise of providing timely, up-to-date and high-quality guidance to target users. Elie Akl et al. describe key issues in living guidelines, including when living guidelines are appropriate, workflows, linkages with living systematic reviews, thresholds for changing recommendations, and approaches to publication and dissemination.
      In this paper, we provide an introduction to LSRs, including a working definition and thoughts on when this approach to review currency may be appropriate. We give an overview of how to undertake an LSR and introduce key considerations, including searching, updating scenarios, production processes, editorial and peer review, and publication. Subsequent papers in this series will describe other important aspects of LSR in more detail (see Box 2).

      3. What is a living systematic review?

      We define an LSR as a systematic review that is continually updated, incorporating relevant new evidence as it becomes available. In practice, this means continual surveillance for new research evidence through ongoing or frequent searches and the inclusion of relevant new information into the review in a timely manner so that the findings of the systematic review remain current.
      LSR is an approach to review updating, not a formal review methodology, and can be applied to any type of review. The frequency of searching and the time taken to include new information into the review are critical to achieving the currency of an LSR but are not specified in this definition. Just as there are pragmatic limits on identifying “all” research in a systematic review search, there are many logistical hurdles to achieving optimal review currency and so pragmatic solutions must be found that balance currency with feasibility. Also, end user needs should guide the definition of optimal “currency”, but empirical evidence of this is currently lacking. In the interim, we propose that LSRs should incorporate relevant new information within a maximum of 6 months of the information becoming available. We expect this proposed upper limit to reduce over time as production and publication systems evolve. In practice, most current LSR pilot projects aim to search most sources at least monthly and make the results of these searches visible to end users within another month.
      In contrast to rapid reviews where methods may be modified to support faster production [
      • Tricco A.C.
      • Antony J.
      • Zarin W.
      • Strifler L.
      • Ghassemi M.
      • Ivory J.
      • et al.
      A scoping review of rapid review methods.
      ,
      • Tricco A.C.
      • Zarin W.
      • Antony J.
      • Hutton B.
      • Moher D.
      • Sherifali D.
      • et al.
      An international survey and modified Delphi approach revealed numerous rapid review methods.
      ], LSRs use standard systematic review methods [
      • Higgins J.
      • Green S.
      Cochrane handbook for systematic reviews of interventions.
      ]. LSRs differ from frequent, standard review updating by an explicit and a priori commitment to keeping the systematic review as current as possible with a predetermined frequency of search and review updating. While it may seem that LSRs resemble the most frequent standard review updates, there are three important differences. First, the explicit and a priori commitment to a predetermined frequency of review updating gives important predictability to end users, reducing their need to plan or undertake their own updating activities. Second, LSRs aim to push the limits of systematic review currency, supporting more dynamic LSRs in the future that are updated immediately whenever new evidence emerges and benefiting other approaches to updating. Third, the commitment to a predetermined frequency of updating enables important downstream opportunities, including living guidelines.

      4. When is an LSR appropriate?

      Not all systematic reviews require updating nor the use of an LSR approach. For a subset of reviews in which updating is appropriate, an LSR approach can be considered. Building on the recently published guidance on systematic review updating by Garner et al. [
      • Garner P.
      • Hopewell S.
      • Chandler J.
      • MacLehose H.
      • Akl E.
      • Beyene J.
      • et al.
      When and how to update systematic reviews: consensus and checklist.
      ], we suggest the following three criteria should guide the decision to initiate an LSR:
      • 1.
        The systematic review is a priority for decision-making. With current review production and publication systems, LSRs are only appropriate when the question is of sufficient importance to decision-making that the allocation of ongoing resources is worthwhile. Over time, we expect the scope of questions in which an LSR approach is appropriate to expand.
      • 2.
        Certainty in the existing evidence is low or very low. An LSR will be useful when the current body of evidence does not provide certainty, and new information is likely to change the findings of the systematic review.
      • 3.
        There is likely to be new research evidence. LSRs are most appropriate when the research field covered by the systematic review is moving relatively quickly, and new evidence is emerging.
      Embarking on an LSR is not a life sentence. It will be appropriate to cease this form of updating when the conditions specified above no longer hold. Further work will be required to inform how these thresholds should be determined.

      5. How to perform an LSR?

      LSR is a process of systematic review updating that uses standard systematic review methods. There are, however, several important implications of this approach for review production and publication, outlined below.

      5.1 Protocol development

      The protocol for an LSR should describe explicitly the approach to searching, including search frequency for each source, and the decision framework and timing for integration of new information into the review (see below and Box 3). It should define the statistical approach to updating of any meta-analyses (Simmonds et al., this issue) and describe the process of transitioning to an LSR. The specific thresholds for transitioning out of an LSR can be defined a priori if known. Other sections of the protocol should provide the usual information appropriate for a systematic review protocol.
      Additional considerations for a living systematic review protocol
      • Background
        • A clear justification for the review being maintained as a living systematic review.
      • Objectives
        • An explicit statement that the review will be maintained as a living systematic review.
      • Searching
        • A description of the frequency of searching for all sources.
      • Screening
        • A description of the frequency of screening, ideally matched to the frequency of searching.
      • Data synthesis
        • A statement describing whether new evidence will be immediately incorporated into the review or may be deferred under certain circumstances.
        • If the incorporation of new evidence may be deferred, a description of the decision framework that will be applied to inform this decision.
        • A description of the planned statistical approach to updating of any meta-analyses.
      • Other
        • A description of any systematic review enablers (such as machine learning or citizen science) used in the review.
        • A statement of the frequency with which the team will regularly review the review scope and methods, including search methods and strategies.
        • Specific thresholds for transitioning out of a Living systematic review mode, if known.
      An LSR protocol should always reflect the conduct of the review, and over time changes to the methods of the review may be appropriate. If the review team decides to modify the methods of the LSR, these changes should be documented in the LSR and in a modified, accessible review protocol. If the review is registered in the PROSPERO register of systematic reviews, it may be appropriate for this listing to be updated.

      5.2 Producing an up-to-date systematic review

      Reaching the “plateau” of LSR updating is only possible once the “mountain” of an up-to-date standard systematic review has been climbed. Thus, the first step in conducting an LSR is to produce an up-to-date (“baseline”) systematic review, either a new review or a standard update of a pre-existing review. For new systematic reviews, a protocol should be published first (Fig. 1), including LSR-specific considerations as described above; the first version of the review should be published; and the review transitioned into an LSR mode. For existing reviews, an updated protocol and standard review update are required, but the order is flexible.
      Figure thumbnail gr1
      Fig. 1Update scenarios. LSR, Living systematic review.

      5.3 Transition to an LSR

      In any of the scenarios described above, it is important that the review be transitioned to an LSR from the time of the standard publication or update. Searches and other aspects of the LSR (see below) therefore need to be underway prior to the standard review publication or update. The work of transitioning to an LSR mode is reduced if this process is initiated soon after the last search of the standard publication or update.

      5.4 Searching

      LSRs use standard systematic review search methods, so while alternative evidence surveillance methods may contribute to an awareness of relevant new information, the currency of LSRs should not be accompanied by a reduction in the sensitivity of the core search strategy. For many bibliographic databases, auto alerts can be set up to provide a regular feed of new citations. In other databases, no automated alert option is available and a manual search needs to be run regularly.
      The most significant challenge for sensitive and frequent searching is trial and other registries, websites, and other unpublished material. At present, these sources require manual, bespoke searching, which may limit the frequency at which they feasibly can be searched. As an example, LSRs could be supported with monthly searches (auto and manual) of major bibliographic databases and three monthly searches of other sources. The specific strategy for an LSR should be developed in consultation with an information specialist.

      5.5 Update scenarios

      Each instance of search will result in reports (e.g., titles and abstracts) that require screening. Standard systematic review methods are applied, so if screening identifies reports for inclusion, risk of bias assessment, data extraction, and incorporation of data into meta-analyses or narrative syntheses (textual approaches to evidence synthesis) are conducted as appropriate. The aim is to update the review as quickly as feasible. However, systematic review updating is still a cumbersome, manual process, and so, pragmatic decisions to defer the inclusion of some new information may be appropriate (Fig. 1). For example, if the new information is likely to have negligible effects on the effect estimates or the certainty of the evidence, it may be appropriate to make the new information visible to end users and defer formal inclusion in the meta-analyses or narrative syntheses of the review. This is a pragmatic solution that should not be necessary in the longer term. In the interim, a transparent approach should be taken to decisions about when to incorporate new information, analogous to frameworks that have been developed to guide the timing of review updating [
      • Garner P.
      • Hopewell S.
      • Chandler J.
      • MacLehose H.
      • Akl E.
      • Beyene J.
      • et al.
      When and how to update systematic reviews: consensus and checklist.
      ]. An important overarching principle is to maximize the value the review provides to end users at all times.
      The pragmatic considerations described above lead to a set of possible updating scenarios. First, if screening of search results yields no new evidence, then it is important to make available to end users the date of last search and the information that no additional evidence was identified.
      Second, if new evidence is identified, a transparent approach should be used to decide whether to incorporate the new evidence into the review, balancing end user needs and pragmatic considerations. End users should be alerted as soon as possible to the date of last search, details of the new evidence, a description of whether the evidence is being incorporated into the review or not, and a transparent justification for this decision.
      Third, whenever it is decided to incorporate new evidence into the review, the next phase of review production is triggered, including risk of bias assessment, data extraction and updating of meta-analyses, narrative syntheses, review findings, conclusions, and implications. During this process, end users should be alerted to the updating process that is underway and the information that is in process of being included. Once completed, the updated LSR should be made available as soon as possible.

      5.6 Production process

      Standard review updating requires the intermittent deployment of substantial resources, whereas maintenance of an LSR requires the continual application of modest resources. Strong project management is necessary to guide the review team in this novel process and manage continual tasks (particularly search and screening) and other tasks required as needed (risk of bias assessment, data extraction, analysis, and write up). Members of the team join, contribute, and leave as needed. This model may maintain more consistent knowledge of the review when compared to the high team turnover often seen between standard updates. However, as with standard updating, consistency of decision-making will be helped by clear documentation of decisions and rationales for these decisions.

      5.7 Enabling technologies

      The efficiency of systematic review production and updating is an important determinant of the feasibility and sustainability of LSRs. Recently, new approaches to review production have emerged that facilitate efficiency, particularly in the context of review production. These include online software platforms, the use of text mining and machine learning, and the contribution of citizen scientists. These “enabling technologies” are discussed further in a later paper in this series (Thomas et al., this issue).

      5.8 Updating of meta-analyses

      All forms of updating commonly require the updating of meta-analyses. For many years, the potential for naïvely repeated meta-analysis to lead to an inflated rate of false positive findings has been debated [
      • Higgins J.P.
      • Whitehead A.
      • Simmonds M.
      Sequential methods for random-effects meta-analysis.
      ]. These issues are explored in another paper in this series (Simmonds et al., this issue).

      5.9 Editorial and peer review

      Importantly, editorial and peer review is required of the LSR protocol and the initial LSR, including the proposed approach to search, the decision framework for inclusion of new evidence into the review, and the statistical methods that will be used for updating of meta-analyses. Subsequent editorial and peer review should be tailored to the updating scenarios described above.
      In the first scenario, no new evidence is identified and the last search date and the number of citations screened are updated. We suggest that in this scenario, no peer review and only review by an editor is required.
      In the second scenario where evidence is identified, but not yet incorporated into the review, editorial review is required of the information that will be updated and available to end users: date of last search, details of new evidence, a description of whether the evidence is being incorporated into the review or not, and a transparent justification for this decision. If this justification is consistent with the approach outlined in the LSR protocol and the methods section of the LSR, we believe peer review is optional and could be initiated at the discretion of the editor.
      In the third scenario, in which new evidence is incorporated into the LSR, this will normally act as a trigger for full editorial and peer review. However, the depth of editorial and peer review should be tailored to the update. If the new evidence has negligible effect on the certainty of the evidence or other findings of the review and makes no difference to review conclusions or implications, we suggest that editorial review alone may be sufficient. If in the view of the editors, the contribution of the new evidence is more substantial, then peer review should be initiated and focused on changes to the LSR since last review.
      Thorough editorial and peer review at the initiation of an LSR and subsequent tailoring of editorial and peer review processes should help to achieve production efficiency and editorial quality. However, we believe additional intermittent review should be incorporated into the LSR lifecycle. For example, LSRs could have a more thorough editorial and peer review once every 1–2 years regardless of whether new evidence had been identified or incorporated, to ensure review methods remain appropriate and review conduct is proceeding as planned and described. Also, postpublication peer review may be particularly relevant to LSRs given the challenge of rapid and frequent peer review in a prepublication peer review model. This could help build a community around the LSR and encourage contributions to the production and use of the review (Thomas et al., this issue). Finally, it is clear that production and editorial platforms that simplify the incorporation of new evidence and target editorial and peer review to the most recent changes improve efficiency and the sustainability of LSRs.

      5.10 Publication

      The dissemination of LSR outputs requires publication formats that can be updated frequently. While this addresses end user needs for up-to-date evidence, it brings several publication challenges. First, the publication platform must allow the publication and linking of review updates, optimally through explicit versioning of the review publication in an online format (e.g., Cochrane Database of Systematic Reviews, F1000 Research) or by using another linking mechanism (e.g., the Crossmark service provided by Crossref).
      Second, authorship of the LSR will evolve over time. As with a clinical trial network, the contribution of each member of the group should be assessed as sufficient for authorship (and meeting ICMJE criteria) or not. As team members leave and their contribution to the current version diminishes, they could be moved from the list of named authors to members of the study group or an acknowledged contributor. In the future, novel ways of acknowledging contributions to the LSR may provide additional value.
      Third, mechanisms should be established to provide timely, frequently updated information (e.g., date of last search, numbers of citations screened, studies awaiting inclusion) to end users without creating significant issues with bibliographic database listing and article metrics (including article citations and “altmetrics”). The number of formal new “publications” generated by the LSR should be neither too high nor too low. Frequent new “publications” (e.g., if a monthly update generated a new publication each time) can lead to an extremely high number of bibliographic database listings and citable publications and a low number of cites per publication. On the other hand, too few publications may erode the perceived academic benefits for the work of maintaining the LSR.
      The key objective of any publication arrangement should be to present information in user friendly and unified formats, maximize the content that is subjected to editorial and peer review, and ensure the model is sufficiently consistent with existing academic incentives to encourage contribution. There are three main options. First, the review can be published using a standard approach with updates published every year or so and more frequently updated information provided through a separate website. The disadvantages of this approach are that the review content is fragmented and a portion of the content may not be subjected to standard editorial and peer review. Second, the review can be published and updated using a standard approach, but with more frequently updated information presented alongside the published articles such that these more frequent changes do not trigger a new digital object identifier (DOI), database listing, or citable publication. This has similar disadvantages to a separate website, but the content is more unified. Third, content can be published and updated frequently in a more unified form, triggering new DOIs, database listings, and citable publications, but arrangements can be established to group each of these metrics (e.g., F1000 Research).

      5.11 Links to other “living” activities

      LSRs do not exist in a vacuum and have important “upstream” and “downstream” implications. The clearest opportunity is for LSRs to support the maintenance of living guidelines. In this model, guidelines are no longer updated as a whole; instead, individual recommendations are updated whenever there is significant change in the underlying evidence base. These opportunities are discussed in a later article in this series (Akl et al., this issue). In addition, LSRs may have important implications for clinical decision support systems, health technology management, knowledge translation, and health policy making. These issues remain largely unexplored at this time. Similarly, LSRs may encourage stronger connections between research production, evidence synthesis, and health decision-making given the long-term, ongoing nature of LSRs and associated opportunities for coproduction by a community of contributors.

      6. Conclusion

      LSRs are a novel approach to systematic review updating that aims to break the current trade-off between methodological rigor and currency, and deliver evidence syntheses that are trustworthy and up to date. In pursuing the development and scale-up of an LSR approach to updating, we seek to dramatically reduce the time for new research to translate into health practice and impact, reduce the waste of society's investment in research, and help all involved in health make sense of the deluge of research. While the fundamentals of the approach are clear and a number of systematic review teams are now converting reviews (both Cochrane and non-Cochrane) into LSRs [
      • Créquit P.
      • Trinquart L.
      • Yavchitz A.
      • Ravaud P.
      Wasted research when systematic reviews fail to provide a complete and up-to-date evidence synthesis: the example of lung cancer.
      ,
      • Cnossen M.
      • Scholten A.
      • Lingsma H.
      • Synnot A.
      • Tavender E.
      • Gantner D.
      • et al.
      Adherence to guidelines in adult patients with traumatic brain injury: a living systematic review.
      ,
      Living Systematic Review Network
      Cochrane living systematic reviews: interim guidance for pilots (Version 0.3, 21 April 2017).
      ,
      • Akl E.A.
      • Kahale L.A.
      • Ballout R.A.
      • Barba M.
      • Yosuico V.E.
      • van Doormaal F.F.
      • et al.
      Parenteral anticoagulation in ambulatory patients with cancer.
      ,
      • Rahal A.K.
      • Badgett R.G.
      • RM H.
      Screening coverage needed to reduce Mortality from Prostate cancer: a living systematic review.
      ], the approach is still new and much remains to be developed and evaluated (Box 4). We welcome others to this endeavor. More information can be found on the LSR Network website at www.cochrane.org/lsr.
      Principles guiding the development of living systematic reviews
      • Keep the end user in mind, maximizing the utility of the review at all stages
      • Minimize additional workload for authors, peer reviewers, editors, and publishers
      • Maximize visibility of the latest findings for the reader
      • Maximize efficiencies through technology and the crowd
      • Streamline workflows and editorial processes
      • Do not reinvent the wheel but build on existing processes and platforms
      • Focus on workable, not perfect, solutions
      • Remain flexible to incorporate new developments in the broader evidence ecosystem

      References

        • Macleod M.
        • Michie S.
        • Roberts I.
        • Dirnagl U.
        • Chalmers I.
        • Ioannidis J.
        • et al.
        Biomedical research: increasing value, reducing waste.
        Lancet. 2014; 383: 101-104
        • Higgins J.
        • Lasserson T.
        • Chandler J.
        • Tovey D.
        • Churchill R.
        Methodological expectations of cochrane intervention reviews.
        Cochrane, London2016
        • Moher D.
        • Liberati A.
        • Tetzlaff J.
        • Altman D.
        The PRISMA Group. Preferred reporting Items for systematic reviews and meta-analyses: the PRISMA statement.
        PLoS Med. 2009; 6: e1000097
        • Institute of Medicine
        Eden J. Finding what works in health care: standards for systematic reviews. National Acadamies Press, Washington, DC2011
        • Tricco A.C.
        • Brehaut J.
        • Chen M.H.
        • Moher D.
        Following 411 Cochrane protocols to completion: a retrospective cohort study.
        PLoS One. 2008; 3: e3684
        • Bastian H.
        • Glasziou P.
        • Chalmers I.
        Seventy-five trials and eleven systematic reviews a day: how will we ever keep up?.
        PLoS Med. 2010; 7: e1000326
        • Shojania K.G.
        • Sampson M.
        • Ansari M.T.
        • Ji J.
        • Doucette S.
        • Moher D.
        How quickly do systematic reviews go out of date? A survival analysis.
        Ann Intern Med. 2007; 147: 224-233
        • Garner P.
        • Hopewell S.
        • Chandler J.
        • MacLehose H.
        • Akl E.
        • Beyene J.
        • et al.
        When and how to update systematic reviews: consensus and checklist.
        BMJ. 2016; 354: i3507
        • Créquit P.
        • Trinquart L.
        • Yavchitz A.
        • Ravaud P.
        Wasted research when systematic reviews fail to provide a complete and up-to-date evidence synthesis: the example of lung cancer.
        BMC Med. 2016; 14: 8
        • Elliott J.H.
        • Turner T.
        • Clavisi O.
        • Thomas J.
        • Higgins J.P.
        • Mavergames C.
        • et al.
        Living systematic reviews: an emerging opportunity to narrow the evidence-practice gap.
        PLoS Med. 2014; 11: e1001603
        • Tricco A.C.
        • Antony J.
        • Zarin W.
        • Strifler L.
        • Ghassemi M.
        • Ivory J.
        • et al.
        A scoping review of rapid review methods.
        BMC Med. 2015; 13: 224
        • Tricco A.C.
        • Zarin W.
        • Antony J.
        • Hutton B.
        • Moher D.
        • Sherifali D.
        • et al.
        An international survey and modified Delphi approach revealed numerous rapid review methods.
        J Clin Epidemiol. 2016; 70: 61-67
        • Higgins J.
        • Green S.
        Cochrane handbook for systematic reviews of interventions.
        John Wiley & Sons, Chichester, UK2008
        • Higgins J.P.
        • Whitehead A.
        • Simmonds M.
        Sequential methods for random-effects meta-analysis.
        Stat Med. 2011; 30: 903-921
        • Cnossen M.
        • Scholten A.
        • Lingsma H.
        • Synnot A.
        • Tavender E.
        • Gantner D.
        • et al.
        Adherence to guidelines in adult patients with traumatic brain injury: a living systematic review.
        J Neurotrauma. 2016; ([Epub ahead of print]. https://doi.org/10.1089/neu.2015.4121.)
        • Living Systematic Review Network
        Cochrane living systematic reviews: interim guidance for pilots (Version 0.3, 21 April 2017).
        Cochrane, London, UK2017 (Available at http://community.cochrane.org/sites/default/files/uploads/inline-files/Transform/LSR Interim guidance_v0.3_20170421.pdf)
        • Akl E.A.
        • Kahale L.A.
        • Ballout R.A.
        • Barba M.
        • Yosuico V.E.
        • van Doormaal F.F.
        • et al.
        Parenteral anticoagulation in ambulatory patients with cancer.
        Cochrane Database Syst Rev. 2014; : CD006652
        • Rahal A.K.
        • Badgett R.G.
        • RM H.
        Screening coverage needed to reduce Mortality from Prostate cancer: a living systematic review.
        PLoS One. 2016; 11: e0153417