Developing trustworthy recommendations as part of an urgent response (1–2 weeks): a GRADE concept paper

  • Elie A. Akl
    Correspondence
    Corresponding author. Department of Internal Medicine, American University of Beirut Medical Center, P.O. Box 11-0236/CRI (E15), Riad-El-Solh Beirut 1107 2020, Beirut, Lebanon. Tel.: 961 1 374374..
    Affiliations
    Department of Health Research Methods, Evidence, and Impact, McMaster University, McMaster University Medical Centre, 1280 Main Street West, Hamilton, Ontario L8S 4K1, Canada
    Search for articles by this author
  • Rebecca L. Morgan
    Affiliations
    Department of Health Research Methods, Evidence, and Impact, McMaster University, McMaster University Medical Centre, 1280 Main Street West, Hamilton, Ontario L8S 4K1, Canada
    Search for articles by this author
  • Andrew A. Rooney
    Affiliations
    National Institute of Environmental Health Science, Durham, NC, USA
    Search for articles by this author
  • Brandiese Beverly
    Affiliations
    National Institute of Environmental Health Science, Durham, NC, USA
    Search for articles by this author
  • Srinivasa Vittal Katikireddi
    Affiliations
    MRC/CSO Social & Public Health Sciences Unit, University of Glasgow, 200 Renfield Street, Glasgow G2 3AX, UK
    Search for articles by this author
  • Arnav Agarwal
    Affiliations
    Department of Medicine, University of Toronto, C. David Naylor Building, 6 Queen’s Park Crescent West, Third Floor, Toronto, ON M5S 3H2, Canada
    Search for articles by this author
  • Brian S. Alper
    Affiliations
    Department of Family and Community Medicine, Medical Knowledge Office, EBSCO Information Services, University of Missouri-Columbia School of Medicine, 10 Estes Street, Ipswich, MA 01938, USA
    Search for articles by this author
  • Carlos Alva-Diaz
    Affiliations
    Universidad Científica del Sur, Facultad de Ciencias de la Salud, Panamericana, Sur km 19, Villa, Lima, Peru
    Search for articles by this author
  • Laura Amato
    Affiliations
    Department of Epidemiology, Lazio Regional Health Services Rome, via Cristoforo Colombo 112, 00147 Rome, Italy
    Search for articles by this author
  • Mohammed T. Ansari
    Affiliations
    School of Epidemiology and Public Health, Faculty of Medicine, University of Ottawa, Room 101, 600 Peter Morand Crescent, Ottawa, Ontario K1G 5Z3, Canada
    Search for articles by this author
  • Jan Brozek
    Affiliations
    Department of Health Research Methods, Evidence, and Impact, McMaster University, McMaster University Medical Centre, 1280 Main Street West, Hamilton, Ontario L8S 4K1, Canada
    Search for articles by this author
  • Derek K. Chu
    Affiliations
    Department of Health Research Methods, Evidence, and Impact, McMaster University, Rm 3V49 McMaster University Medical Centre, Hamilton, ON L8S 4K1, Canada

    Department of Medicine, Rm 3V49 McMaster University Medical Centre, Hamilton, ON L8S 4K1, Canada

    The Research Institute of St. Joe’s Hamilton, Rm 3V49 McMaster University Medical Centre, Hamilton, ON L8S 4K1, Canada
    Search for articles by this author
  • Philipp Dahm
    Affiliations
    Department of Urology, Minneapolis VA Healthcare System, Urology Section, University of Minnesota, One Veterans Drive, Minneapolis, MN 55417, USA
    Search for articles by this author
  • Andrea J. Darzi
    Affiliations
    Department of Health Research Methods, Evidence, and Impact, McMaster University, McMaster University Medical Centre, 1280 Main Street West, Hamilton, Ontario L8S 4K1, Canada
    Search for articles by this author
  • Maicon Falavigna
    Affiliations
    Hospital Moinhos de Vento, Porto Alegre, Brazil
    Search for articles by this author
  • Gerald Gartlehner
    Affiliations
    Department for Evidence-based Medicine and Evaluation, RTI International, Danube University, Dr. Karl Dorrekstrasse, Krems Austria, 3040 East Cornwallis Rd, Research Triangle Park, NC, USA
    Search for articles by this author
  • Hector Pardo-Hernandez
    Affiliations
    Iberoamerican Cochrane Centre, Sant Pau Biomedical Research Institute (IIB Sant Pau), CIBER de Epidemiología y Salud Pública (CIBERESP), Carrer Sant Antoni Maria Claret, 167, Pavelló 18, Planta Baja (Ground Floor), 08025 Barcelona, Spain
    Search for articles by this author
  • Valerie King
    Affiliations
    Center for Evidence-based Policy, Oregon Health & Science University, 3030 South Moody Ave., Suite 250, Portland, OR 97201, USA
    Search for articles by this author
  • Jitka Klugarová
    Affiliations
    Czech National Centre for Evidence-Based Healthcare and Knowledge Translation, Institute of Biostatistics and Analyses, Faculty of Medicine, Masaryk University, Brno, Czech Republic
    Search for articles by this author
  • M.W. Miranda Langendam
    Affiliations
    Amsterdam UMC, University of Amsterdam, Epidemiology and Data Science, Amsterdam Institute of Public Health, Meibergdreef 9, Amsterdam, the Netherlands
    Search for articles by this author
  • Craig Lockwood
    Affiliations
    JBI, School of Public Health, University of Adelaide, Level 3/55 King William Road, North Adelaide 5005, Australia
    Search for articles by this author
  • Manoj Mammen
    Affiliations
    Department of Medicine, State University of New York at Buffalo, 100 High Street, Buffalo, NY, USA
    Search for articles by this author
  • Alexander G. Mathioudakis
    Affiliations
    Division of Infection, Immunity and Respiratory Medicine, School of Biological Sciences, The University of Manchester, Manchester Academic Health Science Centre, 2nd Floor, Education and Research Centre, Wythenshawe Hospital, Manchester University NHS Foundation Trust, Southmoor Road, Manchester M23 9LT, UK

    The North West Lung Centre, Wythenshawe Hospital, Manchester University NHS Foundation Trust, 2nd Floor, Education and Research Centre, Wythenshawe Hospital, Manchester University NHS Foundation Trust, Southmoor Road, Manchester M23 9LT, UK
    Search for articles by this author
  • Michael McCaul
    Affiliations
    Division of Epidemiology and Biostatistics, Department of Global Health, Stellenbosch University, Teaching Building, Faculty of Medicine and Health Sciences, Francie van Zijl Drive, Bellville, Western Cape, South Africa
    Search for articles by this author
  • Joerg J. Meerpohl
    Affiliations
    Institute for Evidence in Medicine, Medical Center & Faculty of Medicine, University of Freiburg, Breisacher Strasse 153, 79110 Freiburg, Germany

    Cochrane Germany, Cochrane Germany Foundation, Breisacher Strasse 153, 79110 Freiburg, Germany
    Search for articles by this author
  • Silvia Minozzi
    Affiliations
    Department of Biomedical Sciences for Health, University of Milan, Milan, Italy
    Search for articles by this author
  • Reem A. Mustafa
    Affiliations
    Department of Internal Medicine, University of Kansas Medical Center, 3901 Rainbow Blvd, MS3002, Kansas City, KS 66160, USA

    Department of Health Research Methods, Evidence, and Impact, McMaster University, 3901 Rainbow Blvd, MS3002, Kansas City, KS 66160, USA
    Search for articles by this author
  • Francesco Nonino
    Affiliations
    IRCCS Istituto delle Scienze Neurologiche di Bologna, UOSI Epidemiologia e Statistica, IRCCS-ISNB, Ospedale Bellaria, Via Altura, 3, 40139 Bologna, Italy
    Search for articles by this author
  • Thomas Piggott
    Affiliations
    Department of Health Research Methods, Evidence, and Impact, McMaster University, McMaster University Medical Centre, 1280 Main Street West 2C Area, Hamilton, Ontario L8S 4K1, Canada
    Search for articles by this author
  • Amir Qaseem
    Affiliations
    American College of Physicians, 190 N Independence Mall West, Philadelphia, PA 19087, USA
    Search for articles by this author
  • John Riva
    Affiliations
    Department of Family Medicine, Department of Health Research Methods, Evidence, and Impact, McMaster University, McMaster University Medical Centre, 1280 Main Street West, Hamilton, Ontario L8S 4K1, Canada
    Search for articles by this author
  • Rachel Rodin
    Affiliations
    Public Health Agency of Canada, 785 Carling Avenue, Ottawa, Ontario, Canada
    Search for articles by this author
  • Nigar Sekercioglu
    Affiliations
    Department of Health Research Methods, Evidence, and Impact, McMaster University, McMaster University Medical Centre, 1280 Main Street West, Hamilton, Ontario L8S 4K1, Canada
    Search for articles by this author
  • Nicole Skoetz
    Affiliations
    Department I of Internal Medicine and Center for Integrated Oncology Aachen Bonn Cologne Duesseldorf, Evidence-based Oncology, Faculty of Medicine and University Hospital Cologne, University of Cologne, Kerpener Str. 15, 50935 Cologne, Germany
    Search for articles by this author
  • Gregory Traversy
    Affiliations
    Public Health Agency of Canada, Ottawa, Ontario, Canada
    Search for articles by this author
  • Kris Thayer
    Affiliations
    U.S. Environmental Protection Agency, Office of Research and Development (ORD, Center for Public Health and Environmental Assessment (CPHEA), Mail Code: B243-01, Building: Bldg B (Room B210I), Research Triangle Park, NC 27711, USA
    Search for articles by this author
  • Holger Schünemann
    Affiliations
    Department of Health Research Methods, Evidence, and Impact, McMaster University, McMaster University Medical Centre, 1280 Main Street West, Hamilton, Ontario L8S 4K1, Canada
    Search for articles by this author
Open AccessPublished:September 30, 2020DOI:https://doi.org/10.1016/j.jclinepi.2020.09.037

      Highlights

      • The proposed approach for developing trustworthy recommendations as part of urgent responses (1–2 week) builds on the general principles of trustworthy guideline development.
      • The approach proposes the following steps: (1) assess the level of urgency; (2) clarify and focus the scope of question(s); (3) prioritize and collect the information needed; (4) assess the adequacy of identified information; and then (5) develop recommendations;
      • Developing recommendations can consist of (1) adopting existing recommendations; (2) adapting existing recommendations; (3) developing new recommendations using existing systematic reviews; or (4) developing new recommendations using identified information and panel input.

      Abstract

      Objectives

      The aim of this study is to propose an approach for developing trustworthy recommendations as part of urgent responses (1–2 week) in the clinical, public health, and health systems fields.

      Study Design and Setting

      We conducted a review of the literature, outlined a draft approach, refined the concept through iterative discussions, a workshop by the Grading of Recommendations Assessment, Development and Evaluation Rapid Guidelines project group, and obtained feedback from the larger Grading of Recommendations Assessment, Development and Evaluation working group.

      Results

      A request for developing recommendations within 2 week is the usual trigger for an urgent response. Although the approach builds on the general principles of trustworthy guideline development, we highlight the following steps: (1) assess the level of urgency; (2) assess feasibility; (3) set up the organizational logistics; (4) specify the question(s); (5) collect the information needed; (6) assess the adequacy of identified information; (7) develop the recommendations using one of the 4 potential approaches: adopt existing recommendations, adapt existing recommendations, develop new recommendations using existing adequate systematic review, or develop new recommendations using expert panel input; and (8) consider an updating plan.

      Conclusion

      An urgent response for developing recommendations requires building a cohesive, skilled, and highly motivated multidisciplinary team with the necessary clinical, scientific, and methodological expertise; adapting to shifting needs; complying with the principles of transparency; and properly managing conflicts of interest.

      Keywords

      What is new?

         Key findings

      • We propose an approach for developing trustworthy recommendations as part of an urgent response (1–2 weeks).

         What this adds to what was known?

      • The proposed alternatives are: adopting or adapting existing recommendations; using existent systematic reviews to develop new recommendations; and, relying on expert panel input to develop new recommendations.

         What is the implication and what should change now?

      • It is critical to leverage collaborations, capabilities, and resources that allow for rapid assessment of adequate information to support recommendation development.

      1. Background

      The development of practice guideline recommendations requires a complex interplay of many participants and involves various tasks including setting priorities, scoping the remit of the guideline, identifying an expert panel group, and managing conflicts of interest [
      • Schunemann H.J.
      • Wiercioch W.
      • Etxeandia I.
      • Falavigna M.
      • Santesso N.
      • Mustafa R.
      • et al.
      Guidelines 2.0: systematic development of a comprehensive checklist for a successful guideline enterprise.
      ,
      • Qaseem A.
      • Forland F.
      • Macbeth F.
      • Ollenschlager G.
      • Phillips S.
      • van der Wees P.
      • et al.
      Guidelines International Network: toward international standards for clinical practice guidelines.
      ]. The most time-consuming tasks are the identification, synthesis, assessment, and presentation of the evidence in ways that allow the guideline groups to formulate recommendations. Over the last 2 decades, the Grading of Recommendations Assessment, Development and Evaluation (GRADE) working group has developed a systematic and transparent approach to creating evidence-informed practice guidelines that facilitate this. The method has been applied across the clinical, public health, and health systems fields [
      • Alonso-Coello P.
      • Oxman A.D.
      • Moberg J.
      • Brignardello-Petersen R.
      • Akl E.A.
      • Davoli M.
      • et al.
      GRADE Evidence to Decision (etd) frameworks: a systematic and transparent approach to making well informed healthcare choices. 2: Clinical practice guidelines.
      ,
      • Moberg J.
      • Oxman A.D.
      • Rosenbaum S.
      • Schünemann H.J.
      • Guyatt G.
      • Flottorp S.
      • et al.
      The GRADE Evidence to Decision (EtD) framework for health system and public health decisions.
      ,
      • Neumann I.
      • Brignardello-Petersen R.
      • Wiercioch W.
      • Carrasco-Labra A.
      • Cuello C.
      • Akl E.
      • et al.
      The GRADE evidence-to-decision framework: a report of its testing and application in 15 international guideline panels.
      ].
      Some situations raise the need for urgent recommendations to support the interventions of clinicians, public health practitioners, and policymakers, for example, to address the COVID-19 pandemic [
      • Thayer K.A.
      • Schunemann H.J.
      Using GRADE to respond to health questions with different levels of urgency.
      ,
      • Schunemann H.J.
      • Hill S.R.
      • Kakad M.
      • Bellamy R.
      • Uyeki T.M.
      • Hayden F.G.
      • et al.
      WHO Rapid Advice Guidelines for pharmacological management of sporadic human infection with avian influenza A (H5N1) virus.
      ]. Under such circumstances, developing trustworthy recommendations in a sufficiently short timeframe is essential but can be challenging in rapidly changing contexts. Groups producing guidelines specifically need to balance the need for developing a timely response with the need to ensure the trustworthiness of their advice.
      The World Health Organization (WHO) defined two types of guidelines developed in response to an emergency or urgent need: emergency (rapid response) guidelines (produced within hours to days) and rapid advice guidelines [
      • Vernooij R.W.
      • Sanabria A.J.
      • Sola I.
      • Alonso-Coello P.
      • Martinez Garcia L.
      Guidance for updating clinical practice guidelines: a systematic review of methodological handbooks.
      ]. Thayer and Schunemann [
      • Thayer K.A.
      • Schunemann H.J.
      Using GRADE to respond to health questions with different levels of urgency.
      ] defined four levels of urgency for developing recommendations: ultra-short emergency response (1–2 hours), urgent response (1–2 weeks), rapid response (1–3 months), and routine response (more than 3 months) (Box 1). Although more detailed advice exists for routine [
      • Schunemann H.J.
      • Wiercioch W.
      • Etxeandia I.
      • Falavigna M.
      • Santesso N.
      • Mustafa R.
      • et al.
      Guidelines 2.0: systematic development of a comprehensive checklist for a successful guideline enterprise.
      ] and rapid responses, so far there is no formal guidance on how to apply GRADE in situations requiring urgent responses [
      • Thayer K.A.
      • Schunemann H.J.
      Using GRADE to respond to health questions with different levels of urgency.
      ,
      • Garritty C.M.
      • Norris S.L.
      • Moher D.
      Developing WHO rapid advice guidelines in the setting of a public health emergency.
      ].
      Levels of urgency for developing recommendations
      Tabled 1
      Ultra-short emergency response: 1–2 h

      Urgent response: 1–2 wk

      Rapid response: up to 3 mo

      Routine response: more than 3 mo
      The objective of this paper is to propose an approach for developing trustworthy recommendations as part of urgent responses (1–2 weeks) in the clinical, public health, and health systems fields.

      2. Methods

      The target audience for the proposed approach are guideline developers. We consider the timeframe from the perspective of the guideline developer, that is, the starting point for the approach is the receipt of a request for developing recommendations (if applicable), while the finishing point is the submission of guidance to the requesting organization. We use the WHO definition of guidelines as “systematically developed evidence-based statements which assist providers, recipients and other stakeholders to make informed decisions about appropriate health interventions” [
      World Health Organization
      Guidelines for WHO guidelines.
      ].
      We developed this approach as members of the GRADE Rapid Guidelines project group. The GRADE Rapid Guidelines project group includes 31 members of the GRADE working group (gradeworkinggroup.org). These members collectively have expertise in preclinical, clinical, and epidemiological primary research, in modeling, in evidence synthesis (including systematic reviews and rapid reviews), and in trustworthy practice guideline development. This expertise spans the breadth fields of clinical, public health, and health policy. The development of the approach started in May 2016, and was based on reviewing selected relevant research (some of which was conducted by members of the GRADE Rapid Guidelines project group) [
      • Thayer K.A.
      • Schunemann H.J.
      Using GRADE to respond to health questions with different levels of urgency.
      ,
      • Florez I.D.
      • Morgan R.L.
      • Falavigna M.
      • Kowalski S.C.
      • Zhang Y.
      • Etxeandia-Ikobaltzeta I.
      • et al.
      Development of rapid guidelines: 2. A qualitative study with WHO guideline developers.
      ,
      • Kowalski S.C.
      • Morgan R.L.
      • Falavigna M.
      • Florez I.D.
      • Etxeandia-Ikobaltzeta I.
      • Wiercioch W.
      • et al.
      Development of rapid guidelines: 1. Systematic survey of current practices and methods.
      ,
      • Morgan R.L.
      • Florez I.
      • Falavigna M.
      • Kowalski S.
      • Akl E.A.
      • Thayer K.A.
      • et al.
      Development of rapid guidelines: 3. GIN-McMaster Guideline Development Checklist extension for rapid recommendations.
      ], iterative discussions within the project group reflecting on the members’ experiences, informal consensus process, and feedback from the larger GRADE working group.
      In terms of the review of selected relevant research, we first considered the findings of a 2018 systematic survey of methods manuals and published guidelines that were developed in a shortened timeframe [
      • Kowalski S.C.
      • Morgan R.L.
      • Falavigna M.
      • Florez I.D.
      • Etxeandia-Ikobaltzeta I.
      • Wiercioch W.
      • et al.
      Development of rapid guidelines: 1. Systematic survey of current practices and methods.
      ]. The review found that rapid guidelines were usually commissioned to address emergencies, rapid increases in the incidence or severity of a condition emerging new evidence for a specific treatment. Overall, there was no consensus on the methods for developing rapid guidance or on the timeframe for development, ranging from 1 to 13 months, when reported. Second, we considered the findings of an interview of guideline developers about key aspects for rapid guideline development [
      • Florez I.D.
      • Morgan R.L.
      • Falavigna M.
      • Kowalski S.C.
      • Zhang Y.
      • Etxeandia-Ikobaltzeta I.
      • et al.
      Development of rapid guidelines: 2. A qualitative study with WHO guideline developers.
      ]. Third, we discussed published and unpublished examples of recommendations developed in response to rapid or urgent needs, with a focus on understanding how standard processes were modified [
      • Schunemann H.J.
      • Hill S.R.
      • Kakad M.
      • Bellamy R.
      • Uyeki T.M.
      • Hayden F.G.
      • et al.
      WHO Rapid Advice Guidelines for pharmacological management of sporadic human infection with avian influenza A (H5N1) virus.
      ,
      • Morgan R.L.
      • Florez I.
      • Falavigna M.
      • Kowalski S.
      • Akl E.A.
      • Thayer K.A.
      • et al.
      Development of rapid guidelines: 3. GIN-McMaster Guideline Development Checklist extension for rapid recommendations.
      ,
      World Health Organization
      Clinical management of severe acute respiratory infection when novel coronavirus (2019-nCoV) infection is suspected: interim guidance, 28 January 2020.
      ,
      • Schünemann H.J.
      • Wiercioch W.
      • Brozek J.
      • Etxeandia-Ikobaltzeta I.
      • Mustafa R.A.
      • Manja V.
      • et al.
      GRADE Evidence to Decision (EtD) frameworks for adoption, adaptation, and de novo development of trustworthy recommendations: grade-adolopment.
      ,
      World Health Organization
      Infection Prevention and Control guidance for Long-Term Care Facilities in the context of COVID-19: interim guidance, 21 March 2020.
      ].
      We debated, revised, and refined the concepts to be included in the approach based on iterative discussions during conference calls and a workshop by the project group (Rome, Italy, April 27, 2017). Members of the project group also drew on their experiences with developing recommendations in response to rapid or urgent needs [
      • Bhimraj A.
      • Morgan R.L.
      • Shumaker A.H.
      • Lavergne V.
      • Baden L.
      • Cheng V.C.
      • et al.
      Infectious Diseases Society of America Guidelines on the treatment and management of patients with COVID-19.
      ,
      • Sultan S.
      • Lim J.K.
      • Altayar O.
      • Davitkov P.
      • Feuerstein J.D.
      • Siddique S.M.
      • et al.
      AGA institute rapid recommendations for gastrointestinal procedures during the COVID-19 pandemic.
      ,
      • Qaseem A.
      • Yost J.
      • Etxeandia-Ikobaltzeta I.
      • Miller M.C.
      • Abraham G.M.
      • Obley A.J.
      • et al.
      Should clinicians use chloroquine or hydroxychloroquine alone or in combination with azithromycin for the prophylaxis or treatment of COVID-19?.
      ]. Finally, the GRADE working group approved this paper using its expedited publication approval process, developed in light of the COVID-19 pandemic, and the need for development of trustworthy recommendations for its management internationally as part of urgent responses.

      3. Results

      The project group agreed to build the urgent response approach on the commonly accepted principles of standard guideline development (e.g., conflict of interest management) [
      • Schunemann H.J.
      • Wiercioch W.
      • Etxeandia I.
      • Falavigna M.
      • Santesso N.
      • Mustafa R.
      • et al.
      Guidelines 2.0: systematic development of a comprehensive checklist for a successful guideline enterprise.
      ,
      • Qaseem A.
      • Forland F.
      • Macbeth F.
      • Ollenschlager G.
      • Phillips S.
      • van der Wees P.
      • et al.
      Guidelines International Network: toward international standards for clinical practice guidelines.
      ]. The group also agreed that the urgent response approach can also build on the GIN-McMaster guideline development checklist extension for rapid recommendations (see table here: https://bit.ly/rapidGDC) [
      • Florez I.D.
      • Morgan R.L.
      • Falavigna M.
      • Kowalski S.C.
      • Zhang Y.
      • Etxeandia-Ikobaltzeta I.
      • et al.
      Development of rapid guidelines: 2. A qualitative study with WHO guideline developers.
      ]. Ideally, the process should include the conduct of a rapid review, whenever feasible. However, and given the major challenges with feasibility, in the subsequent sections we did not consider a rapid review as an option.
      The proposed approach assumes adherence to the criteria for applying or using GRADE [
      Cardiac involvement in a patient with coronavirus disease 2019 (COVID-19).
      ]. These criteria include giving explicit consideration to the GRADE factors for determining the direction and strength of a recommendation and, ideally, using GRADE Evidence to Decision (EtD) frameworks and tables [
      • Alonso-Coello P.
      • Schünemann H.J.
      • Moberg J.
      • Brignardello-Petersen R.
      • Akl E.A.
      • Davoli M.
      • et al.
      GRADE Evidence to Decision (EtD) frameworks: a systematic and transparent approach to making well informed healthcare choices. 1: Introduction.
      ]. Under standard guideline development conditions, the EtD table is populated by evidence (typically from a systematic review) for a number of factors (the health effects of the interventions of interest, values and preferences, certainty of evidence, resource use, impact on equity, acceptability, and feasibility). The panelists review the evidence for each factor, then make judgments for those factors in relation to the interventions of interest, before deciding on the strength and direction for the recommendation [
      • Alonso-Coello P.
      • Schünemann H.J.
      • Moberg J.
      • Brignardello-Petersen R.
      • Akl E.A.
      • Davoli M.
      • et al.
      GRADE Evidence to Decision (EtD) frameworks: a systematic and transparent approach to making well informed healthcare choices. 1: Introduction.
      ].
      In terms of sources of evidence, systematic reviews may not be feasible given the timeframe of 1–2 weeks for the complete recommendation development process. If, in spite of this tight timeframe, the guideline developers find it feasible to conduct a rapid review, the reviewers could refer to the Interim Guidance from the Cochrane Rapid Reviews Methods Group [
      • Garritty C.
      • Gartlehner G.
      • Kamel C.
      • King V.
      • Nussbaumer-Streit B.
      • Stevens A.
      Interim guidance from the cochrane rapid reviews methods group.
      ]. In addition, the guideline developers may consider a timely publication of the systematic review, including coordination with the coordinating organization/funder, reviewers, and journal [
      Clinical characteristics of children with coronavirus disease 2019 in Hubei, China.
      ]. The proposed approach offers the alternatives listed at the bottom of Fig. 1, which are based on the use of existing guidelines and existent systematic reviews (discussed in more detail below).
      Figure thumbnail gr1
      Fig. 1Steps involved in urgent guideline development.
      The following paragraphs discuss the steps of the urgent response approach and assume that existing and adequate guidelines and systematic reviews, but not new rapid reviews, are being considered. However, if a rapid review is performed it may be added at step 5 in this process. We provide illustrative examples of urgent situations related to environmental exposures and to the COVID-19 pandemic.

      3.1 Assess the level of urgency

      The purpose of this first step is to confirm that the response in developing the recommendation should be an “urgent” one, as opposed to “emergent” or “rapid” (see Box 1). Although there is no tool specifically developed for that purpose, one can use tools such as the WHO rapid risk assessment of acute public health events [
      ]. Risk assessment includes the assessment of the likelihood of the occurrence of a hazard causing an adverse event, as well as the likely magnitude of the consequences of the event over a specified period.

      3.2 Assess feasibility

      It is important for the guideline developers to assess the feasibility of meeting the “urgent” timeframe (i.e., 1–2 weeks). Aspects to be considered include the following: (1) the number and complexity of questions and comparisons; (2) the expected amount of work needed for collecting the literature; (3) the number of team members available to contribute to the different tasks; (4) sources of funding that can be promptly tapped to support the urgent response; (5) the availability of adequate coordination capacity; (6) the panelists who can be summoned to contribute to collecting the needed information, panel discussion, and peer review in a very intensive manner; and (7) the political and institutional support to expedite the administrative process—collaboratively with the relevant endorsing bodies (particularly the approval of the final product and if needed, of the dissemination and implementation strategies), if applicable.

      3.3 Set up the organizational logistics

      As the guideline developers start working with the requestors on specifying the questions (see next section), it is important to set up the organizational logistics for the project. Forming a steering group with executive power that would meet frequently (e.g., at least on a daily basis) to ensure the progress of the project and the compliance with the timeline is key. Similarly, the steering group should involve methodologists with prior experience in developing recommendations as part of urgent or rapid responses. Also, in the context of international collaboration, having members from different time zones can help ensure the ability to sustain project workflow.
      Starting on day 1, the steering group should identify and quickly recruit members of the different groups (including methodologists, panelists, reviewers) and secure their solid commitment to the expected tasks with a clear timeline and schedule. However, it might be challenging to recruit stakeholders (particularly patients’ representatives) in the context of a rapid response (e.g., in the middle of a pandemic). Building on existing networks of stakeholders or having a roster of stakeholders willing to serve as panelists when needed can be helpful. The aim should be meaningful and equitable multi-stakeholder engagement [
      • Petkovic J.
      • Riddle A.
      • Akl E.A.
      • Khabsa J.
      • Lytvyn L.
      • Atwere P.
      • et al.
      Protocol for the development of guidance for stakeholder engagement in health and healthcare guideline development and implementation.
      ].
      Similarly, when applicable, the steering group needs to prospectively liaise closely with the organization’s oversight and quality committee to ensure compliance with quality requirements. The steering group should start as early as possible drafting the final report and fill in its different sections as the information becomes available, to expedite its submission and later publication. Other important logistics include a streamlined communication plan with the different groups involved, with the goal of minimizing email burden; and reliable and easy to use document sharing and online meeting platforms. Ideally, the organization should have in place a “preparedness plan” to mount an urgent response for developing recommendations (with the appropriate processes and tools).

      3.4 Specify the question(s)

      To make the process feasible, the guideline developers should work with the requestor to ensure the guideline’s scope is reasonably narrow. In addition, the guideline developers should ensure the questions are specific and well defined, for example, using the PICO (Population, Intervention, Comparator, Outcome) or PECO (Population, Exposure, Comparator, Outcome) frameworks for interventions and exposures respectively [
      • Morgan R.L.
      • Whaley P.
      • Thayer K.A.
      • Schunemann H.J.
      Identifying the PECO: a framework for formulating good questions to explore the association of environmental and other exposures with health outcomes.
      ,
      • Guyatt G.H.
      • Oxman A.D.
      • Kunz R.
      • Atkins D.
      • Brozek J.
      • Vist G.
      • et al.
      GRADE guidelines: 2. Framing the question and deciding on important outcomes.
      ]. Even within the PICO or PECO frameworks, additional specificity (narrowing the scope) when defining the population, intervention, or exposure may facilitate the urgent development process. Annex 1 provides a brief description and examples of the elements of the PICO and PECO questions.
      Prioritizing outcomes is an essential step of the guideline development process. However, it can be time consuming and not feasible under urgent conditions. Ideally, there would be a “ready to use” standardized set of outcomes for the condition under consideration (e.g., COVID-19 Core Outcomes initiative) [
      Analysis of epidemiological and clinical features in older patients with Corona Virus Disease 2019 (COVID-19) out of Wuhan.
      ]. When such a set is not available, the guideline developers could rely on previous prioritization efforts. For example, the Surviving Sepsis Campaign (SSC) guidelines on the management of critically ill adults with COVID-19, instead of performing a new prioritization of outcomes, used the outcome prioritization informed by the ongoing SSC guideline 2020 work and expert panel input [
      • Alhazzani W.
      • Moller M.H.
      • Arabi Y.M.
      • Loeb M.
      • Gong M.N.
      • Fan E.
      • et al.
      Surviving sepsis campaign: guidelines on the management of critically ill adults with Coronavirus Disease 2019 (COVID-19).
      ]. The group could also rely on outcomes prioritized for a similar disease (e.g., influenza or severe acute respiratory syndrome [SARS]). In some urgent situations, the group might opt to prioritize only one critical outcome. For example, slowing the spread of the pandemic (“flattening the curve”) was initially considered as the sole critical outcome when considering public health interventions for the COVID-19 pandemic.
      Urgent situations are typically associated with scarcity of data, particularly for clinical outcomes (e.g., clinical recovery). In such cases, the guideline developers might need to rely on surrogate outcomes (e.g., viral load) for informing recommendations. Indirect evidence can be used to inform other components of the PICO or PECO question. For example, the National Institute for Health and Care Excellence (NICE)’s “interim process and methods for developing rapid guidelines on COVID-19” calls for using information related to SARS, Middle East respiratory syndrome (MERS), and pandemic influenza [
      Clinical observation and management of COVID-19 patients.
      ].
      It is important to determine a priori what indirect evidence would be considered and used, if needed. Similar key decisions in the design and conduct of the project (e.g., eligibility criteria) should ideally be done a priori. However, and given the limited time to fully consider the different aspects of the project, post hoc decisions are more likely in urgent compared with routine response when developing guidelines.

      3.5 Collect the information needed

      As mentioned earlier, the proposed approach considers using existing guidelines and systematic reviews and does not necessarily require the conduct of new rapid reviews. NICE has adopted such an approach in its “interim process and methods for developing rapid guidelines on COVID-19” [
      • Alhazzani W.
      • Moller M.H.
      • Arabi Y.M.
      • Loeb M.
      • Gong M.N.
      • Fan E.
      • et al.
      Surviving sepsis campaign: guidelines on the management of critically ill adults with Coronavirus Disease 2019 (COVID-19).
      ]. They aimed to reuse existing guidance as much as possible, without conducting systematic literature searches. They first searched for published guidance, including WHO COVID-19 guidance, NICE guidance, and guidance from professional organizations, both inside and outside the UK [
      • Alhazzani W.
      • Moller M.H.
      • Arabi Y.M.
      • Loeb M.
      • Gong M.N.
      • Fan E.
      • et al.
      Surviving sepsis campaign: guidelines on the management of critically ill adults with Coronavirus Disease 2019 (COVID-19).
      ]. If no guidance was identified, they searched for and prioritized the following types of publications, in the following order: systematic reviews, randomized controlled trials, and published expert opinions. Many organizations use a similar approach. For example, in 2008, the European Food Safety Authority relied on a previously published systematic review to develop its urgent guidance related to the presence of melamine in composite food products [
      European Food Safety Authority
      Statement of EFSA on risks for public health due to the presence of melamine in infant milk and other milk products in China.
      ].
      In addition to information about the health effects, the guideline developers may need to identify contextual information. In the example of mass evacuation in response to a nuclear incident, there is a need to collect information about the cost and feasibility of such a major undertaking, as well as the acceptability by relevant stakeholders (e.g., governmental agencies, citizens). This information may be available from administrative databases (e.g., for cost), and through consulting with community leaders (e.g., to assess the acceptability by stakeholders).
      In terms of information sources, the published literature may not be the only optimal source in urgent and rapidly developing situations, like the COVID-19 pandemic. Information sources may include preprint servers, open research databases, and research data repositories. However, such sources may not include conventional processes of peer review and may require additional critical appraisal or caution in the interpretation of the accuracy or stability of the data. Specific to environmental exposure scenarios, it may be important to evaluate or model the evidence for health effects when little is known about the exposure (e.g., a chemical with uncertain effects).

      3.6 Assess the adequacy of identified information

      Once identified, the published literature (guidelines, systematic reviews, and if applicable, primary studies of any design) needs to be assessed for “adequacy” based on the three following criteria: relevance, credibility, and currency [
      • Darzi A.
      • Harfouche M.
      • Arayssi T.
      • Alemadi S.
      • Alnaqbi K.A.
      • Badsha H.
      • et al.
      Adaptation of the 2015 American College of Rheumatology treatment guideline for rheumatoid arthritis for the Eastern Mediterranean Region: an exemplar of the GRADE Adolopment.
      ].
      • Relevance or directness, according to GRADE terminology [
        • Guyatt G.H.
        • Oxman A.D.
        • Kunz R.
        • Woodcock J.
        • Brozek J.
        • Helfand M.
        • et al.
        GRADE guidelines: 8. Rating the quality of evidence--indirectness.
        ], refers to the extent to which the population, exposure/intervention, and outcomes identified in the literature (whether for the original recommendation or for the systematic reviews) reflect the ones in the question being addressed by the recommendation. If any items are different enough that we would expect different relative effects, then the evidence is considered indirect.
      • Credibility reflects the quality of conduct of the guideline or of the systematic review being considered for use in the process of the urgent response. There are a number of tools that could be used for that purpose: the advancing guideline development, reporting and evaluation in health care II to assess the quality of conduct of guidelines [
        • Brouwers M.C.
        • Kho M.E.
        • Browman G.P.
        • Burgers J.S.
        • Cluzeau F.
        • Feder G.
        • et al.
        AGREE II: advancing guideline development, reporting and evaluation in health care.
        ]; the assessment of multiple systematic reviews-2 to assess the methodological quality of systematic reviews [
        • Shea B.J.
        • Reeves B.C.
        • Wells G.
        • Thuku M.
        • Hamel C.
        • Moran J.
        • et al.
        AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both.
        ], and ROBIS to assess the risk of bias of systematic reviews [
        • Whiting P.
        • Savovic J.
        • Higgins J.P.
        • Caldwell D.M.
        • Reeves B.C.
        • Shea B.
        • et al.
        ROBIS: a new tool to assess risk of bias in systematic reviews was developed.
        ].
      • Currency or recency: although the currency of a systematic review depends on its search date, the currency of a guideline depends similarly to a large extent on the search dates of the systematic reviews used to inform recommendations. In some cases, even if the search date is not current, a systematic review may still be up to date if no new evidence was published after the date of the search; however, a search would need to be conducted to verify this assumption. Rapidly updating a published systematic review may be efficient, if that review is credible and new evidence is known to have been published since the date of the original search [
        • Garner P.
        • Hopewell S.
        • Chandler J.
        • MacLehose H.
        • Schunemann H.J.
        • Akl E.A.
        • et al.
        When and how to update systematic reviews: consensus and checklist.
        ].
      The guideline developers do not need to conduct the above assessment for all identified relevant literature (guidelines, systematic reviews, and primary studies). For example, if they end up adopting an existing recommendation (addressed in step 6), although the recommendation under consideration needs to be assessed, this would not apply to all other identified literature (e.g., systematic reviews and primary studies).
      The adequacy of identified literature is not an “all or none” assessment. For example, the guideline developer might identify three potentially relevant systematic reviews. Each one of the three may be deficient in relation to one of the three above criteria, but to different extents. We suggest considering the criteria in the following order: relevance, credibility, and currency. This means that the guideline developers would select a relevant but outdated systematic review over an updated, but less relevant review. The idea is that it is more efficient to “fix” the former review with a quick update than to use the latter less relevant review.
      During the 2014 Elk River chemical spill, the most relevant information on the toxicology of the crude 4-methylcyclohexanemethanol (MCHM) was obtained directly from the manufacturer [
      National Toxicology Program
      National Toxicology Program research project: West Virginia chemical spill. Board of Scientific Counselors December 9–10, 2014 meeting.
      ]. Within 6 days, the Scientific Review Panel for the National Library of Medicine’s Hazardous Substance Data Bank was able to expedite the evaluation of the toxicology, chemical composition, and chemical purpose to assess the credibility of the information [
      National Toxicology Program
      National Toxicology Program research project: West Virginia chemical spill. Board of Scientific Counselors December 9–10, 2014 meeting.
      ]. This early evaluation was a critical step in the process of understanding the risks of exposure and the best course of action for cleanup. However, additional research was needed to better understand the short-term adverse health effects associated with MCHM, and no additional information was available at the time. To expand the knowledge base for MCHM and other chemicals associated with the spill with the most current data, the Centers for Disease Control and Prevention and the Agency for Toxic Substances Disease Registry enlisted the help of the National Toxicology Program to conduct a series of short duration studies [
      National Toxicology Program
      Nomination Summary for West Virginia Elk River chemical spill (N21408).
      ]. The results obtained from these studies assisted the West Virginia Bureau of Public Health in responding to public health concerns related to the MCHM and other chemical exposures that occurred during the incident.

      3.7 Develop the recommendations

      Depending on the availability and adequacy of identified literature (guidelines or systematic reviews), the guideline developers may develop recommendations using one of the four potential approaches: (1) adopt existing recommendations; (2) adapt existing recommendations; (3) develop new recommendations using existing systematic and/or rapid reviews [
      • Schünemann H.J.
      • Wiercioch W.
      • Brozek J.
      • Etxeandia-Ikobaltzeta I.
      • Mustafa R.A.
      • Manja V.
      • et al.
      GRADE Evidence to Decision (EtD) frameworks for adoption, adaptation, and de novo development of trustworthy recommendations: grade-adolopment.
      ]; or (4) develop new recommendations using evidence provided by panelists (also referred to as expert input or expert evidence) [
      • Schunemann H.J.
      • Zhang Y.
      • Oxman A.D.
      Expert Evidence in Guidelines G
      Distinguishing opinion from evidence in guidelines.
      ].

      3.7.1 Adopt vs. adapt existing recommendations

      The literature search may find an original recommendation that is ‘adequate’ for the urgent response, i.e., is relevant, credible and current. At that point, the expert panel group should decide whether to adopt the original recommendation (i.e., use it as is) or adapt it. Practically, the expert panel group ‘adopts’ a recommendation when it does not change either the direction or the strength of that recommendation. Otherwise, the recommendation is ‘adapted’.
      To decide on adopting vs. adapting an original recommendation, the expert panel group needs to assess whether the direction or the strength of recommendation may be affected by any difference in the following factors (ideally using a decision-making framework, e.g., EtD table) [
      • Alonso-Coello P.
      • Schünemann H.J.
      • Moberg J.
      • Brignardello-Petersen R.
      • Akl E.A.
      • Davoli M.
      • et al.
      GRADE Evidence to Decision (EtD) frameworks: a systematic and transparent approach to making well informed healthcare choices. 1: Introduction.
      ]:
      • Rating of the importance of outcomes: a change in the rating of the importance of outcomes might lead to a change in the judgment of the balance of benefits and harms. For example, an urgent response recommendation addressing the use of invasive ventilation in patients with COVID-19 might reflect a different prioritization of the importance of “patient survival” outcome relative to “transmission of infection to healthcare workers” outcome (in comparison to the original recommendation) [
        • Schunemann H.J.
        • Khabsa J.
        • Solo K.
        • Khamis A.M.
        • Brignardello-Petersen R.
        • El-Harakeh A.
        • et al.
        Ventilation techniques and risk for transmission of coronavirus disease, including COVID-19.
        ].
      • Indirectness: what could have been judged as direct evidence in the context of the original recommendation might be judged as indirect evidence in the urgent response context (e.g., effectiveness of treatment with corticosteroids among patients with MERS-CoV vs. COVID-19) population. This might lead to rating down the certainty of evidence. One could use the GRADE framework to judge whether that situation warrants rating down the certainty of evidence for indirectness [
        • Guyatt G.H.
        • Oxman A.D.
        • Kunz R.
        • Woodcock J.
        • Brozek J.
        • Helfand M.
        • et al.
        GRADE guidelines: 8. Rating the quality of evidence--indirectness.
        ].
      • Rating of evidence: the expert panel group may rate the certainty of evidence differently than the original recommendation for reasons other than indirectness judgment (see above). Examples include different judgment of the risk of bias or imprecision, and consideration of new or emerging evidence.
      • Baseline risks: the absolute reduction of an outcome (e.g., mortality in laboratorians or other staff with a potential occupational risk for Ebola receiving a pre-exposure Ebola vaccination) might be higher or lower in the context of the urgent response (e.g., Ebola outbreak responders) compared to context of the original recommendation (e.g., due to a difference in baseline risk of mortality between the two contexts). This could lead to a change in balance of benefits and harms.
      • Perspective: the original recommendation might have been developed from an individual or clinical perspective, while the urgent response has a population or public health perspective. The new perspective may broaden the relevant stakeholders (e.g., beyond the patients, their caregivers, and healthcare providers), and increase the weight of EtD criteria such as cost effectiveness and impact on equity.
      • Contextual factors: the judgment on contextual factors (such as resource use, impact on equity, acceptability, and feasibility) might change when considering the context of the rapid response (in comparison to the original context).
      For each of the above factors, the judgment (between the urgent guidance setting and the original one) needs to be sufficiently different for panel members to adapt the recommendation (i.e., modify its direction and/or strength). The recommendation statement wording may be edited to enhance the usability for the intended target group.
      WHO developed interim guidance for the “clinical management of severe acute respiratory infection when COVID-19 infection is suspected” in consultation with the International Forum for Acute Care Trialists, International Severe Acute Respiratory and Emerging Infection Consortium, and SSC [
      World Health Organization
      Clinical management of severe acute respiratory infection when novel coronavirus (2019-nCoV) infection is suspected: interim guidance, 28 January 2020.
      ]. Guideline developers originally adapted this interim guidance from “Clinical management of severe acute respiratory infection when Middle East respiratory syndrome coronavirus (MERS-CoV) infection is suspected: interim guidance” [
      World Health Organization
      Clinical management of severe acute respiratory infection when Middle East respiratory syndrome coronavirus (MERS-CoV) infection is suspected: interim guidance.
      ]. In the first edition of COVID-19 guidance, developers adopted many document sections, research questions, and applicable guidance verbatim, adapting some directives to reflect underlying uncertainty about the microbiological profile of COVID-19 when informed from the indirect information from SARS (caused by SARS-CoV1) and MERS (caused by MERS-CoV) cases [
      World Health Organization
      Clinical management of severe acute respiratory infection when novel coronavirus (2019-nCoV) infection is suspected: interim guidance, 28 January 2020.
      ].

      3.7.2 Develop new recommendation using existing adequate systematic review

      One other reasonable scenario would be to find an adequate systematic review that would jump-start the development of de novo recommendation(s). The development would follow the standard GRADE evidence assessment and recommendation development process with some potential shortcuts to ensure the process is completed within the desired timeframe [
      • Schunemann H.J.
      • Wiercioch W.
      • Etxeandia I.
      • Falavigna M.
      • Santesso N.
      • Mustafa R.
      • et al.
      Guidelines 2.0: systematic development of a comprehensive checklist for a successful guideline enterprise.
      ].
      The Infectious Diseases Society of America developed rapid recommendations on the treatment and management of hospitalized patients with COVID-19, including a recommendation on corticosteroid treatment for hospitalized patients with acute COVID-19 [
      • Bhimraj A.
      • Morgan R.L.
      • Shumaker A.H.
      • Lavergne V.
      • Baden L.
      • Cheng V.C.
      • et al.
      Infectious Diseases Society of America Guidelines on the treatment and management of patients with COVID-19.
      ]. At the time of the first iteration of the guideline, the review team did not identify any direct evidence to inform this recommendation; however, they identified a systematic review reporting on corticosteroid use among patients with SARS-CoV-1 or MERS-CoV [
      • Stockman L.J.
      • Bellamy R.
      • Garner P.
      SARS: systematic review of treatment effects.
      ]. The guideline panel determined this existing review to be direct enough to inform their recommendation.

      3.7.3 Develop new recommendation using expert panel input

      When no adequate recommendations are available for adopting or adapting, and no adequate systematic reviews are available to inform a de novo recommendation, using expert panel input as the sole source of evidence is a feasible alternative for an urgent response. It is important to clarify that panel input is still important when developing the recommendation based on existing recommendations or systematic reviews.
      Fig. 2 shows the steps involved in developing a new recommendation using expert panel input. The panelists are asked to review the literature they are provided with and then fill out a “panelist EtD” table [
      • Alonso-Coello P.
      • Schünemann H.J.
      • Moberg J.
      • Brignardello-Petersen R.
      • Akl E.A.
      • Davoli M.
      • et al.
      GRADE Evidence to Decision (EtD) frameworks: a systematic and transparent approach to making well informed healthcare choices. 1: Introduction.
      ]. In that table, and in lieu of the systematically collected evidence, the panelists provide a description of their “expert evidence” consisting of their observations and experiences (equivalent to case reports and case series) [
      • Schunemann H.J.
      • Zhang Y.
      • Oxman A.D.
      Expert Evidence in Guidelines G
      Distinguishing opinion from evidence in guidelines.
      ]; these are expected to reflect “facts” (as opposed to opinions). In the next step, the steering group collates the input from all the panelists and populates the EtD that will be used as the basis for panel discussion and consensus building. Typically, the panel chair builds consensus with the panelists through discussion, and if needed through (iterative) voting. Alternatively, and for efficiency purposes, the chair can make suggestions and have panelists agree or disagree with them; however, caution is needed to ensure a broad range of perspectives is considered.
      Figure thumbnail gr2
      Fig. 2Steps involved in developing a new recommendation using panel input.

      3.8 Consider an updating plan

      Urgent situations are typically associated not only with scarcity of data but also with rapidly developing evidence base and contextual information. This raises the consideration of establishing an updating plan, ideally through a living process [
      • Vernooij R.W.
      • Sanabria A.J.
      • Sola I.
      • Alonso-Coello P.
      • Martinez Garcia L.
      Guidance for updating clinical practice guidelines: a systematic review of methodological handbooks.
      ,
      • Akl E.A.
      • Meerpohl J.J.
      • Elliott J.
      • Kahale L.A.
      • Schünemann H.J.
      Living systematic reviews: 4. Living guideline recommendations.
      ]. This is particularly relevant when emerging data can potentially lead to a change in the recommendation. The updating plan would need to define the frequency of reassessment of the recommendation (e.g., weekly, monthly), and to be adjusted to the speed of development of the urgent situation and of the emergence of the evidence.
      In the case of an urgency related to an environmental exposure, there might be a need to continuously monitor the level of exposure within the population of interest for the purpose of triggering or updating the recommendations [
      • Akl E.A.
      • Meerpohl J.J.
      • Elliott J.
      • Kahale L.A.
      • Schünemann H.J.
      Living systematic reviews: 4. Living guideline recommendations.
      ]. In the example of the nuclear incident, regular collection of environmental radioactive iodine levels from the field would be needed to reverse the recommendation for mass evacuation.

      4. Discussion

      Developing trustworthy guidelines in a relatively short timeframe are of utmost importance but can be extraordinarily challenging. In this paper, we propose an approach for developing trustworthy recommendations as part of an urgent response (1–2 weeks). The approach offers the alternatives of using existing guidelines to adopt or adapt recommendations; using existent systematic reviews to develop new recommendations; and, when the previous options are not possible, relying on expert panel input to develop new recommendations.
      The alternatives of adopting, adapting, and using existent reviews to developing new recommendations are similar to the ones proposed by the GRADE-ADOLOPMENT methodology [
      • Schunemann H.J.
      • Wiercioch W.
      • Brozek J.
      • Etxeandia-Ikobaltzeta I.
      • Mustafa R.A.
      • Manja V.
      • et al.
      GRADE Evidence to Decision (EtD) frameworks for adoption, adaptation, and de novo development of trustworthy recommendations: grade-adolopment.
      ]. Although that methodology was designed to address guideline development primarily to save and share resources, the approach proposed in this paper is intended to address limited available time (and also potentially limited resources, recognizing that the absence of guideline recommendations may pose more of a threat than rapidly developed ones).
      The alternative relying on expert panel input to develop new recommendations might be perceived as going against the principles of evidence-based medicine. However, Schunemann et al. [
      • Schunemann H.J.
      • Zhang Y.
      • Oxman A.D.
      Expert Evidence in Guidelines G
      Distinguishing opinion from evidence in guidelines.
      ] argued that expert opinion can be considered as “expert evidence.” They did caution however against confusing expert evidence with expert’s views and judgments. Similarly, Djulbegovic and Guyatt [
      • Djulbegovic B.
      • Guyatt G.
      Evidence vs consensus in clinical practice guidelines.
      ] refer to the “experience of individual clinicians” as a type of evidence. They also make the point that evidence requires interpretation and “a consensus process must determine that interpretation” in the context of guidelines.
      Although we may not have captured all situations unique to creating urgent responses, we have prioritized providing a broad, structured, and pragmatic approach. The illustrative examples exemplify how a number of guideline organizations had to rapidly react to urgent situations related to environmental exposures or to the COVID-19 pandemic. Many of the examples highlight how flexibility was needed on the side of the developer in terms of maintaining trustworthy methodology.
      Having recommendations for situations that might recur on an urgent basis may reduce the need for development of recommendations within urgent timeframes. For example, Public Health England has existing guidelines for handling nuclear incidents that are regularly updated, so that this material is available in advance of catastrophic events [
      • Nisbet A.F.
      Public health protection in radiation emergencies.
      ]. Nevertheless, there will continue to be new unexpected incidents that require decisions to be made for which no relevant recommendations are already available or could have been foreseen. In these instances, it is critical to leverage collaborations, capabilities, and resources that allow for rapid assessment of adequate information to support recommendation development. Moreover, it would be important for coordinated efforts by guideline developers to avoid duplication of efforts and sometimes inconsistent or even contradictory recommendations.

      Acknowledgments

      Alexander G. Mathioudakis is supported by the NIHR Manchester Biomedical Research Centre ( BRC ), United Kindgom. Srinivasa V. Katikireddi acknowledges funding from an NRS Senior Clinical Fellowship (SCAF/15/02), the Medical Research Council (MC_UU_12017/13), United Kindgom, and the Scottish Government Chief Scientist Office (SPHSU13), United Kindgom. Derek K. Chu is a CAAIF-CSACI- AllerGen Emerging Clinician-Scientist Research Fellow, supported by the Canadian Allergy, Asthma and Immunology Foundation ( CAAIF ), Canada, the Canadian Society of Allergy and Clinical Immunology (CSACI), Canada, and AllerGen NCE Inc. (the Allergy, Genes and Environment Network supported by the Networks of Centres of Excellence), Canada.

      Supplementary data

      Annex 1: Elements of the PICO and PECO questions The use of the PICO vs. the PECO framework is dependent on the knowledge gap being addressed. For instance, when little is known about the outcomes arising from the exposure to a potential hazard, a PECO question is needed. Conversely, if the exposure is known to be a hazard, and information is needed about the effectiveness and safety of a mitigating intervention, a PICO question is needed [

      • Guyatt G.H.
      • Oxman A.D.
      • Kunz R.
      • Atkins D.
      • Brozek J.
      • Vist G.
      • et al.
      GRADE guidelines: 2. Framing the question and deciding on important outcomes.
      ].
      • Population: Includes those whose health outcomes might be affected (positively or negatively) by either the exposure in a PECO question (e.g., exposed to the released radioactivity during a nuclear incident), or the management options in a PICO question (e.g., mass evacuations or providing iodine following a nuclear incident). For the COVID-19 pandemic five populations of interest have been described: people known to be infected, people presumed to be infected, people exposed to those infected, people not known to be exposed or infected, and people who have recovered from infection [
        • Fineberg H.V.
        Ten weeks to crush the curve.
        ].
      • Exposure/Intervention and comparator(s): These terms refer to either classifying the potential hazard (exposure) or to classifying the management options (interventions) under consideration. In some cases, there is a need to define the exposure level above which an action would be triggered and/or below which an action could be discontinued. It is important to note that interventions may consist of a comprehensive, coordinated response consisting of many components. Consequently, it is important to clarify the specific components, timing and duration of their delivery, and the individuals involved [
        • Guyatt G.H.
        • Oxman A.D.
        • Kunz R.
        • Atkins D.
        • Brozek J.
        • Vist G.
        • et al.
        GRADE guidelines: 2. Framing the question and deciding on important outcomes.
        ]. In the nuclear incident example, when considering the intervention of providing iodine vs. not providing iodine, the following needed to be specified: the dosage and number of doses of prophylactic iodine, and the timeframe within which iodine should be ingested.
      • Outcomes: These should ideally include all critical and important health outcomes that the alternative management options are expected to influence impact. In the above example, these could include the health and psychosocial effects of radioactive iodine exposure, and the benefits and adverse effects of prophylactic iodine therapy. Health-related outcomes should be prioritized based on patient importance. In some cases, policy makers might be interested in the impact on nonhealth outcomes, for example, socioeconomic impact of mass evacuation and road accidents caused by mass evacuation

      References

        • Schunemann H.J.
        • Wiercioch W.
        • Etxeandia I.
        • Falavigna M.
        • Santesso N.
        • Mustafa R.
        • et al.
        Guidelines 2.0: systematic development of a comprehensive checklist for a successful guideline enterprise.
        CMAJ. 2014; 186: E123-E142
        • Qaseem A.
        • Forland F.
        • Macbeth F.
        • Ollenschlager G.
        • Phillips S.
        • van der Wees P.
        • et al.
        Guidelines International Network: toward international standards for clinical practice guidelines.
        Ann Intern Med. 2012; 156: 525-531
        • Alonso-Coello P.
        • Oxman A.D.
        • Moberg J.
        • Brignardello-Petersen R.
        • Akl E.A.
        • Davoli M.
        • et al.
        GRADE Evidence to Decision (etd) frameworks: a systematic and transparent approach to making well informed healthcare choices. 2: Clinical practice guidelines.
        BMJ. 2016; 353: i2089
        • Moberg J.
        • Oxman A.D.
        • Rosenbaum S.
        • Schünemann H.J.
        • Guyatt G.
        • Flottorp S.
        • et al.
        The GRADE Evidence to Decision (EtD) framework for health system and public health decisions.
        Health Res Pol Syst. 2018; 16: 45
        • Neumann I.
        • Brignardello-Petersen R.
        • Wiercioch W.
        • Carrasco-Labra A.
        • Cuello C.
        • Akl E.
        • et al.
        The GRADE evidence-to-decision framework: a report of its testing and application in 15 international guideline panels.
        Implement Sci. 2016; 11: 93
        • Thayer K.A.
        • Schunemann H.J.
        Using GRADE to respond to health questions with different levels of urgency.
        Environ Int. 2016; 92-93: 585-589
        • Schunemann H.J.
        • Hill S.R.
        • Kakad M.
        • Bellamy R.
        • Uyeki T.M.
        • Hayden F.G.
        • et al.
        WHO Rapid Advice Guidelines for pharmacological management of sporadic human infection with avian influenza A (H5N1) virus.
        Lancet Infect Dis. 2007; 7: 21-31
        • Vernooij R.W.
        • Sanabria A.J.
        • Sola I.
        • Alonso-Coello P.
        • Martinez Garcia L.
        Guidance for updating clinical practice guidelines: a systematic review of methodological handbooks.
        Implement Sci. 2014; 9: 3
        • Garritty C.M.
        • Norris S.L.
        • Moher D.
        Developing WHO rapid advice guidelines in the setting of a public health emergency.
        J Clin Epidemiol. 2017; 82: 47-60
        • World Health Organization
        Guidelines for WHO guidelines.
        World Health Organization, Geneva2003
        • Florez I.D.
        • Morgan R.L.
        • Falavigna M.
        • Kowalski S.C.
        • Zhang Y.
        • Etxeandia-Ikobaltzeta I.
        • et al.
        Development of rapid guidelines: 2. A qualitative study with WHO guideline developers.
        Health Res Policy Syst. 2018; 16: 62
        • Kowalski S.C.
        • Morgan R.L.
        • Falavigna M.
        • Florez I.D.
        • Etxeandia-Ikobaltzeta I.
        • Wiercioch W.
        • et al.
        Development of rapid guidelines: 1. Systematic survey of current practices and methods.
        Health Res Policy Syst. 2018; 16: 61
        • Morgan R.L.
        • Florez I.
        • Falavigna M.
        • Kowalski S.
        • Akl E.A.
        • Thayer K.A.
        • et al.
        Development of rapid guidelines: 3. GIN-McMaster Guideline Development Checklist extension for rapid recommendations.
        Health Res Policy Syst. 2018; 16: 63
        • World Health Organization
        Clinical management of severe acute respiratory infection when novel coronavirus (2019-nCoV) infection is suspected: interim guidance, 28 January 2020.
        World Health Organization, 2020 (Available at)
        https://apps.who.int/iris/handle/10665/330893
        Date accessed: October 17, 2020
        • Schünemann H.J.
        • Wiercioch W.
        • Brozek J.
        • Etxeandia-Ikobaltzeta I.
        • Mustafa R.A.
        • Manja V.
        • et al.
        GRADE Evidence to Decision (EtD) frameworks for adoption, adaptation, and de novo development of trustworthy recommendations: grade-adolopment.
        J Clin Epidemiol. 2017; 81: 101-110
        • World Health Organization
        Infection Prevention and Control guidance for Long-Term Care Facilities in the context of COVID-19: interim guidance, 21 March 2020.
        World Health Organization, 2020 (Available at)
        https://apps.who.int/iris/handle/10665/331508
        Date accessed: October 17, 2020
        • Bhimraj A.
        • Morgan R.L.
        • Shumaker A.H.
        • Lavergne V.
        • Baden L.
        • Cheng V.C.
        • et al.
        Infectious Diseases Society of America Guidelines on the treatment and management of patients with COVID-19.
        Clin Infect Dis. 2020;
        • Sultan S.
        • Lim J.K.
        • Altayar O.
        • Davitkov P.
        • Feuerstein J.D.
        • Siddique S.M.
        • et al.
        AGA institute rapid recommendations for gastrointestinal procedures during the COVID-19 pandemic.
        Gastroenterology. 2020;
        • Qaseem A.
        • Yost J.
        • Etxeandia-Ikobaltzeta I.
        • Miller M.C.
        • Abraham G.M.
        • Obley A.J.
        • et al.
        Should clinicians use chloroquine or hydroxychloroquine alone or in combination with azithromycin for the prophylaxis or treatment of COVID-19?.
        Ann Intern Med. 2020;
      1. Cardiac involvement in a patient with coronavirus disease 2019 (COVID-19).
        JAMA Cardiol. 2020;
        • Alonso-Coello P.
        • Schünemann H.J.
        • Moberg J.
        • Brignardello-Petersen R.
        • Akl E.A.
        • Davoli M.
        • et al.
        GRADE Evidence to Decision (EtD) frameworks: a systematic and transparent approach to making well informed healthcare choices. 1: Introduction.
        BMJ. 2016; : 353
        • Garritty C.
        • Gartlehner G.
        • Kamel C.
        • King V.
        • Nussbaumer-Streit B.
        • Stevens A.
        Interim guidance from the cochrane rapid reviews methods group.
        Cochrane Rapid Rev. 2020; 2020
      2. Clinical characteristics of children with coronavirus disease 2019 in Hubei, China.
        Curr Med Sci. 2020; 24
      3. Rapid risk assessment of acute public health events. World Health Organization, 2012 (Available at)
        • Petkovic J.
        • Riddle A.
        • Akl E.A.
        • Khabsa J.
        • Lytvyn L.
        • Atwere P.
        • et al.
        Protocol for the development of guidance for stakeholder engagement in health and healthcare guideline development and implementation.
        Syst Rev. 2020; 9: 21
        • Morgan R.L.
        • Whaley P.
        • Thayer K.A.
        • Schunemann H.J.
        Identifying the PECO: a framework for formulating good questions to explore the association of environmental and other exposures with health outcomes.
        Environ Int. 2018; 121: 1027-1031
        • Guyatt G.H.
        • Oxman A.D.
        • Kunz R.
        • Atkins D.
        • Brozek J.
        • Vist G.
        • et al.
        GRADE guidelines: 2. Framing the question and deciding on important outcomes.
        J Clin Epidemiol. 2011; 64: 395-400
      4. Analysis of epidemiological and clinical features in older patients with Corona Virus Disease 2019 (COVID-19) out of Wuhan.
        Clin Infect Dis. 2020; 25
        • Alhazzani W.
        • Moller M.H.
        • Arabi Y.M.
        • Loeb M.
        • Gong M.N.
        • Fan E.
        • et al.
        Surviving sepsis campaign: guidelines on the management of critically ill adults with Coronavirus Disease 2019 (COVID-19).
        Intensive Care Med. 2020; 46: 854-887
      5. Clinical observation and management of COVID-19 patients.
        Emerg. 2020; 9: 687-690
        • European Food Safety Authority
        Statement of EFSA on risks for public health due to the presence of melamine in infant milk and other milk products in China.
        EFSA J. 2008; 6: 807
        • Darzi A.
        • Harfouche M.
        • Arayssi T.
        • Alemadi S.
        • Alnaqbi K.A.
        • Badsha H.
        • et al.
        Adaptation of the 2015 American College of Rheumatology treatment guideline for rheumatoid arthritis for the Eastern Mediterranean Region: an exemplar of the GRADE Adolopment.
        Health Qual Life Outcomes. 2017; 15: 183
        • Guyatt G.H.
        • Oxman A.D.
        • Kunz R.
        • Woodcock J.
        • Brozek J.
        • Helfand M.
        • et al.
        GRADE guidelines: 8. Rating the quality of evidence--indirectness.
        J Clin Epidemiol. 2011; 64: 1303-1310
        • Brouwers M.C.
        • Kho M.E.
        • Browman G.P.
        • Burgers J.S.
        • Cluzeau F.
        • Feder G.
        • et al.
        AGREE II: advancing guideline development, reporting and evaluation in health care.
        CMAJ. 2010; 182: E839-E842
        • Shea B.J.
        • Reeves B.C.
        • Wells G.
        • Thuku M.
        • Hamel C.
        • Moran J.
        • et al.
        AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both.
        BMJ. 2017; 358: j4008
        • Whiting P.
        • Savovic J.
        • Higgins J.P.
        • Caldwell D.M.
        • Reeves B.C.
        • Shea B.
        • et al.
        ROBIS: a new tool to assess risk of bias in systematic reviews was developed.
        J Clin Epidemiol. 2016; 69: 225-234
        • Garner P.
        • Hopewell S.
        • Chandler J.
        • MacLehose H.
        • Schunemann H.J.
        • Akl E.A.
        • et al.
        When and how to update systematic reviews: consensus and checklist.
        BMJ. 2016; 354: i3507
        • National Toxicology Program
        National Toxicology Program research project: West Virginia chemical spill. Board of Scientific Counselors December 9–10, 2014 meeting.
        (Available at)
        • National Toxicology Program
        Nomination Summary for West Virginia Elk River chemical spill (N21408).
        (Available at)
        • Schunemann H.J.
        • Zhang Y.
        • Oxman A.D.
        • Expert Evidence in Guidelines G
        Distinguishing opinion from evidence in guidelines.
        BMJ. 2019; 366: l4606
        • Schunemann H.J.
        • Khabsa J.
        • Solo K.
        • Khamis A.M.
        • Brignardello-Petersen R.
        • El-Harakeh A.
        • et al.
        Ventilation techniques and risk for transmission of coronavirus disease, including COVID-19.
        Ann Intern Med. 2020;
        • World Health Organization
        Clinical management of severe acute respiratory infection when Middle East respiratory syndrome coronavirus (MERS-CoV) infection is suspected: interim guidance.
        World Health Organization, 2019 (Available at)
        https://apps.who.int/iris/handle/10665/178529
        Date accessed: October 17, 2020
        • Stockman L.J.
        • Bellamy R.
        • Garner P.
        SARS: systematic review of treatment effects.
        PLoS Med. 2006; 3: e343
        • Akl E.A.
        • Meerpohl J.J.
        • Elliott J.
        • Kahale L.A.
        • Schünemann H.J.
        Living systematic reviews: 4. Living guideline recommendations.
        J Clin Epidemiol. 2017; 91: 47-53
        • Schunemann H.J.
        • Wiercioch W.
        • Brozek J.
        • Etxeandia-Ikobaltzeta I.
        • Mustafa R.A.
        • Manja V.
        • et al.
        GRADE Evidence to Decision (EtD) frameworks for adoption, adaptation, and de novo development of trustworthy recommendations: grade-adolopment.
        J Clin Epidemiol. 2017; 81: 101-110
        • Djulbegovic B.
        • Guyatt G.
        Evidence vs consensus in clinical practice guidelines.
        JAMA. 2019; 322: 725-726
        • Nisbet A.F.
        Public health protection in radiation emergencies.
        PHE publication, 2019 (Available at)
        • Fineberg H.V.
        Ten weeks to crush the curve.
        N Engl J Med. 2020; 382: e37