Getting trustworthy guidelines into the hands of decision-makers and supporting their consideration of contextual factors for implementation globally: recommendation mapping of COVID-19 guidelines

  • Tamara Lotfi
    Affiliations
    Department of Health Research Methods, Evidence, and Impact, World Health Organization Collaborating Center for Infectious Diseases, McMaster University, Hamilton, Ontario, Canada

    Michael G. DeGroote Cochrane Canada Centre, McMaster University, Hamilton, Ontario, Canada

    McMaster GRADE Centre, McMaster University, Hamilton, Ontario, Canada
    Search for articles by this author
  • Adrienne Stevens
    Affiliations
    Michael G. DeGroote Cochrane Canada Centre, McMaster University, Hamilton, Ontario, Canada

    McMaster GRADE Centre, McMaster University, Hamilton, Ontario, Canada
    Search for articles by this author
  • Elie A. Akl
    Affiliations
    Department of Medicine, McMaster University, Hamilton, Ontario, Canada

    Clinical Research Institute, Faculty of Medicine, American University of Beirut, Beirut, Lebanon
    Search for articles by this author
  • Maicon Falavigna
    Affiliations
    National Institute for Health Technology Assessment, Federal Univesity of Rio Grande do Sul, PortoAlegre, Brazil

    Hospital Moinhos de Vento, Porto Alegre, Brazil
    Search for articles by this author
  • Tamara Kredo
    Affiliations
    South African Medical Research Council, Cape Town, South Africa
    Search for articles by this author
  • Joseph L. Mathew
    Affiliations
    Postgraduate Institute of Medical Education and Research, Chandigarh, India
    Search for articles by this author
  • Holger J. Schünemann
    Correspondence
    Corresponding author. Tel.: +1 905 525 9140 × 24931
    Affiliations
    Department of Health Research Methods, Evidence, and Impact, World Health Organization Collaborating Center for Infectious Diseases, McMaster University, Hamilton, Ontario, Canada

    Michael G. DeGroote Cochrane Canada Centre, McMaster University, Hamilton, Ontario, Canada

    McMaster GRADE Centre, McMaster University, Hamilton, Ontario, Canada
    Search for articles by this author
  • on behalf of theeCOVID Collaborators
    Author Footnotes
    # Collaborators: Ruby Pawankar, Thomas Piggott, Kevin Pottie, Alexis F. Turgeon, Micayla Matthews, Omar Dewidar, Alfonso Iorio, Waleed Alhazzani, Zahra Saad, Wojtek Wiercioch, Rayane El-khoury, Dominik Mertz, Miranda Langendam, Zachary Munn, Lawrence Mbuagbaw, Zil Nasir, Karla Solo, Heba Hussein, Yuan Chi, Lara Kahale, Joanne Khabsa, Signe Flottorp, Jitka Klugarová, María Ximena Rojas, Gabriel Rada, Miloslav Klugar, Lorenzo Moja, Gunn Vist, Vivian Welch, Elizabeth Ghogomu, Lucia Kantorová, Priyadharshini Ramakrishnan, Derek K. Chu, Stephanie Duda, Ammar Mektebi, Muhammad Ismail Shawish, Maureen Smith.Non-author contributors: Mohamad Tarek Madani, Mark Loeb, Funeka Bango, Megan Thomas, Dina M. Sami Khalifa, Artur Nowak, Lisa Hartling, Joerg J Meerpohl, Marge Reinap, Reem Mustafa, Sarah Scott (NICE), Binu Abraham Philip, Verônica Colpani, Tanja Kuchenmüller, Nayê Balzan Schneider, Stephanie Duda, Romina Brignardello-Petersen, Ashley Motilall, Antonio Bognanni, Sura Issa, Sabrina Price, Pablo Alonso-Coello, Ludovic Reveiz, Pema Gurung
  • Author Footnotes
    # Collaborators: Ruby Pawankar, Thomas Piggott, Kevin Pottie, Alexis F. Turgeon, Micayla Matthews, Omar Dewidar, Alfonso Iorio, Waleed Alhazzani, Zahra Saad, Wojtek Wiercioch, Rayane El-khoury, Dominik Mertz, Miranda Langendam, Zachary Munn, Lawrence Mbuagbaw, Zil Nasir, Karla Solo, Heba Hussein, Yuan Chi, Lara Kahale, Joanne Khabsa, Signe Flottorp, Jitka Klugarová, María Ximena Rojas, Gabriel Rada, Miloslav Klugar, Lorenzo Moja, Gunn Vist, Vivian Welch, Elizabeth Ghogomu, Lucia Kantorová, Priyadharshini Ramakrishnan, Derek K. Chu, Stephanie Duda, Ammar Mektebi, Muhammad Ismail Shawish, Maureen Smith.Non-author contributors: Mohamad Tarek Madani, Mark Loeb, Funeka Bango, Megan Thomas, Dina M. Sami Khalifa, Artur Nowak, Lisa Hartling, Joerg J Meerpohl, Marge Reinap, Reem Mustafa, Sarah Scott (NICE), Binu Abraham Philip, Verônica Colpani, Tanja Kuchenmüller, Nayê Balzan Schneider, Stephanie Duda, Romina Brignardello-Petersen, Ashley Motilall, Antonio Bognanni, Sura Issa, Sabrina Price, Pablo Alonso-Coello, Ludovic Reveiz, Pema Gurung

      Abstract

      Published research on COVID-19 is increasing rapidly and integrated in guidelines. The trustworthiness of guidelines can vary depending on the methods used to assemble and evaluate the evidence, the completeness and transparency of reporting on the process undertaken and how conflicts of interest are addressed.
      With a global consortium of partners and collaborators, we have created a catalogue of COVID-19 recommendations as our direct response to the increased need for structured access to high quality guidance in the field. The COVID19 map of recommendations and gateway to contextualization (https://covid19.recmap.org) is a living project: emerging guideline literature is added on an ongoing basis, allowing granular access to individual recommendations.
      Building on prior work on mapping recommendations for the World Health Organization tuberculosis guidelines, a novel feature of this map is the self-directed contextualization of the recommendations using the GRADE-Adolopment approach to adopt, adapt or synthesize de novo recommendations for context specific questions. Through our map, stakeholders access the evidence underpinning a recommendation, select what needs to be contextualized and go through the steps of development of adapted recommendations. This one-stop shop portal of evidence-informed recommendations, built with intuitive functionalities, easy to navigate and with a support team ready to guide users across the maps, represents a long-needed tool for decision-makers, guideline developers and the public at large.

      Keywords

      What is new?

         Key findings

      • The COVID19 map of recommendations and gateway to contextualization (https://covid19.recmap.org) is a living project: emerging guideline literature is added on an ongoing basis, allowing granular access to individual recommendations that are critically appraised for their trustworthiness.

         What this adds to what is known?

      • We expanded on the only prior recommendation map of WHO tuberculosis recommendations (who.tuberculosis.recmap.org) and added multiple features that are easy to navigate and allows users across the maps to adapt recommendations to their context; all of that focused on clinical, public health and health systems COVID19 recommendations.

         What is the implication/what should change now?

      • Through our map, stakeholders can access the evidence underpinning a recommendation, and adapt it for their context through the “adolopment module” which links the guideline project on the GRADEpro tool to original content, select what needs to be contextualized and go through the steps of development of adapted recommendations.
      Published research on COVID-19 is increasing rapidly [
      • Gazendam A.
      • et al.
      The “Infodemic” of journal publication associated with the novel coronavirus disease.
      ,
      • Palayew A.
      • et al.
      Pandemic publishing poses a new COVID-19 challenge.
      ,
      • Furstenau L.B.
      • et al.
      A bibliometric network analysis of coronavirus during the first eight months of COVID-19 in 2020.
      ,
      • Zhao S.
      • et al.
      A quality evaluation of guidelines on five different viruses causing public health emergencies of international concern.
      ,
      • Luo W.-Y.
      • et al.
      Management in the paediatric wards facing novel coronavirus infection: a rapid review of guidelines and consensuses.
      ]. Clinicians, patients, citizens and others decision-makers are pressured to make urgent decisions and adapt to the rapid and changing demands in clinical and public health [
      • Dagens A.
      • et al.
      Scope, quality, and inclusivity of clinical guidelines produced early in the COVID-19 pandemic: rapid review.
      ]. For both decision-makers and knowledge synthesis (KS) and knowledge translation (KT) experts who package evidence for decision-makers, registries and portals have emerged to catalogue primary studies and syntheses of evidence on COVID-19. Among those are the Norwegian Institute of Public Health living evidence map (NIPH) [], the COVID-19 L•OVE platform [], COVID-END, McMaster's COVID 19 Evidence Alerts [], the Cochrane Library and the Open-Access Data and Computational Resources to address COVID-19 by the National Institute for Health. Moreover, methodologists have supported the global community of KS producers by providing guidance on methodologies relevant to the urgency of COVID-19 [
      • Tricco A.C.
      • Garritty C.M.
      • Boulos L.
      • Lockwood C.
      • Wilson M.
      • McGowan J.
      • et al.
      Rapid review methods more challenging during COVID-19: commentary with a focus on 8 knowledge synthesis steps.
      ,
      • Schünemann H.J.
      • Santesso N.
      • Vist G.E.
      • Cuello C.
      • Lotfi T.
      • Flottorp S.
      • et al.
      Using GRADE in situations of emergencies and urgencies: certainty in evidence and recommendations matters during the COVID-19 pandemic, now more than ever and no matter what.
      ,
      • Garritty C.
      • Gartlehner G.
      • Nussbaumer-Streit B.
      • King V.J.
      • Hamel C.
      • Kamel C.
      • et al.
      Cochrane rapid reviews methods group offers evidence-informed guidance to conduct rapid reviews.
      ].
      Similarly, clinical, public health and health policy guidelines have been developed by various organizations in response to the COVID-19 pandemic [
      • Dagens A.
      • et al.
      Scope, quality, and inclusivity of clinical guidelines produced early in the COVID-19 pandemic: rapid review.
      ]. Notable are those produced by the World Health Organization (WHO) as they consider a broad context when producing evidence-based recommendations, which can be adapted locally, where needed; this adaptation process is an efficient means of supporting decision-making rather than producing de novo guidelines. Particularly in emergencies, WHO undertakes approaches that optimize the quality of guideline development that reduce the biases brought by the need for a rapid process [
      • Norris S.L.
      • Sawin V.I.
      • Ferri M.
      • Sastre L.R.
      • Porgo T.V.
      An evaluation of emergency guidelines issued by the World Health Organization in response to four infectious disease outbreaks.
      ,
      • Schunemann H.J.
      • Hill S.R.
      • Kakad M.
      • Vist G.E.
      • Bellamy R.
      • Stockman L.
      • et al.
      Transparent development of the WHO rapid advice guidelines.
      . When these guidelines address and provide evidence for complementary considerations important for decision-making, such as cost and cost effectiveness, impact on health equity, acceptability, feasibility, their contextualization to other settings becomes possible; this is particularly important during emergencies [
      • Schünemann H.J.
      • Santesso N.
      • Vist G.E.
      • Cuello C.
      • Lotfi T.
      • Flottorp S.
      • et al.
      Using GRADE in situations of emergencies and urgencies: certainty in evidence and recommendations matters during the COVID-19 pandemic, now more than ever and no matter what.
      ]. The GRADE Adolopment [
      • Tugwell P.
      • Knottnerus J.A.
      Adolopment—a new term added to the Clinical Epidemiology Lexicon.
      ,
      • Falzon D.
      • Schünemann H.J.
      • Harausz E.
      • González-Angulo L.
      • Lienhardt C.
      • Jaramillo E.
      • et al.
      World Health Organization treatment guidelines for drug-resistant tuberculosis, 2016 update.
      approach allows the adoption (without modifications) of a recommendation, its adaptation (with modifications) and de novo development when no recommendation exists in the source guideline.
      The GRADE Evidence to Decision (EtD) framework in guideline development facilitates the consideration of criteria that are important for informing a recommendation, each addressed with the best available evidence, where possible. Contextualization of an existing guideline should integrate research or other evidence specific to the particular circumstance, such as a geographic region, for one or more of the above mentioned EtD criteria, e.g. cost or feasibility. This process culminates in a decision to adopt a recommendation as is or to adapt it because of that contextual information [
      • Akl E.A.
      • Morgan R.L.
      • Rooney A.A.
      • Beverly B.
      • Katikireddi S.V.
      • Agarwal A.
      • et al.
      Developing trustworthy recommendations as part of an urgent response (1–2 weeks): a GRADE concept paper.
      ,
      • Darzi A.
      • Harfouche M.
      • Arayssi T.
      • Alemadi S.
      • Alnaqbi K.A.
      • Badsha H.
      • et al.
      Adaptation of the 2015 American College of Rheumatology treatment guideline for rheumatoid arthritis for the Eastern Mediterranean Region: an exemplar of the GRADE Adolopment.
      . It is also possible that the available guideline information is incomplete and de novo development is required.
      The trustworthiness of guidelines can vary depending on the methods used to assemble and evaluate the evidence [
      • Norris S.L.
      • Sawin V.I.
      • Ferri M.
      • Sastre L.R.
      • Porgo T.V.
      An evaluation of emergency guidelines issued by the World Health Organization in response to four infectious disease outbreaks.
      ,
      • Collaboration T.A.
      Development and validation of an international appraisal instrument for assessing the quality of clinical practice guidelines: the AGREE project.
      ,
      • Schunemann H.J.
      • Wiercioch W.
      • Etxeandia I.
      • Falavigna M.
      • Santesso N.
      • Mustafa R.
      • et al.
      Guidelines 2.0: systematic development of a comprehensive checklist for a successful guideline enterprise.
      the completeness and transparency of reporting on the process undertaken [
      • Brouwers M.C.
      • Kerkvliet K.
      • Spithoff K.
      The AGREE Reporting Checklist: a tool to improve reporting of clinical practice guidelines.
      ,
      • Chen Y.
      • Yang K.
      • Marušic A.
      • Qaseem A.
      • Meerpohl J.
      • Flottorp S.
      • et al.
      A reporting tool for practice guidelines in health care: the RIGHT statement.
      and addressing conflicts of interest [
      • Alia E.
      • Grant-Kels J.M.
      The trustworthiness and transparency in clinical practice guidelines versus the ongoing damaging power of direct and indirect conflict of interest.
      ,
      • Guyatt G
      • A.E. Hirsh J
      • Kearon C
      • Crowther M
      • Gutterman D
      • Lewis SZ
      • et al.
      The vexing problem of guidelines and conflict of interest: a potential solution.
      ,
      • Schunemann H.J.
      • Al-Ansary L.A.
      • Forland F.
      • Kersten S.
      • Komulainen J.
      • Kopp I.B.
      • et al.
      Guidelines international network: principles for disclosure of interests and management of conflicts in guidelines.
      ,
      • Schunemann H.J.
      • Osborne M.
      • Moss J.
      • Manthous C.
      • Wagner G.
      • Sicillian L.
      • et al.
      An Official American Thoracic Society policy statement: managing conflict of interest in professional societies.
      ,
      • Boyd E.A.
      • Bero L.A.
      Improving the use of research evidence in guideline development: managing conflicts of interests.
      ], for example, among other standards proposed by the Guidelines International Network and the Institute of Medicine [
      • Amir Qaseem F.F.
      • Macbeth F
      • Ollenschläger G
      • Phillips S
      Philip van der Wees, for the Board of Trustees of the Guidelines International Network*, Guidelines International Network: Toward International Standards for Clinical Practice Guidelines.
      ]. The Appraisal of Guidelines, REsearch and Evaluation II (AGREE II) tool is a long-established instrument used to appraise the quality of clinical guidelines, for example in the evaluation of emerging guidance in COVID-19 [
      • Brouwers M.C.
      • Kho M.E.
      • Browman G.P.
      • Burgers J.S.
      • Cluzeau F.
      • Feder G.
      • et al.
      AGREE II: advancing guideline development, reporting and evaluation in health care.
      ].
      As with previous public health emergencies, the current pandemic has triggered a rapid rise in guideline development to answer decision makers’ questions [
      • Zhao S.
      • et al.
      A quality evaluation of guidelines on five different viruses causing public health emergencies of international concern.
      ]. Of central importance in a quickly shifting COVID-19 decision-making landscape is making that information available while considering the quality of the guidance and contextualization of those recommendations to specific circumstances. Accordingly, our team has capitalized on the innovations and advances in methodological and technological aspects of guideline development to respond to the current pandemic by building a living catalogue of evidence and recommendations (https://covid19.recmap.org) through funding by the Canadian Institutes for Health Research in collaboration with a list of partners (https://covid19.recmap.org/about).
      Our foray into electronic recommendations maps began with earlier work with the WHO's Global Tuberculosis department to build the WHO eTB living map of recommendations (https://who.tuberculosis.recmap.org/). The eTB tool is freely accessible, collects all WHO's recommendations on TB prevention and care, and categorizes them for efficient presentation for different stakeholders [
      • Hajizadeh A
      • L.T. Dennis Falzon F
      • Mertz D
      • Nieuwlaat R
      • Gebreselassie N
      • Jaramillo E
      • et al.
      Recommendation mapping of the World Health Organization's guidelines on tuberculosis: a new approach to digitizing and presenting recommendations.
      ].
      Building on the TB model, we have created a catalogue of COVID-19 recommendations as our direct response to the increased need for a structured access to high quality guidance in the field. The COVID19 map of recommendations and gateway to contextualization (https://covid19.recmap.org) is a living project whereby the emerging guideline literature is added on an ongoing basis, allowing granular access to individual recommendations. This is a product of the collaboration between Cochrane Canada and other Cochrane entities, the WHO Collaborating Center for Infectious Diseases, Research Methods and Recommendations at McMaster University, GRADE centers, the Norwegian Institute of Public Health, the Guidelines International Network, the National Institute of Health and Care Excellence (NICE), WHO/PAHO, and many other institutions or organizations (https://covid19.recmap.org/about).
      Our scope of work is focused on identifying COVID-19 guidelines, assessing their quality, and making their recommendations available for stakeholders (Fig. 1). We provide an ‘adolopment’ module to facilitate contextualization and implementation by decision-makers across the globe. To achieve this, we have developed bibliographic searches that pre-filter citation yields to capture guidelines on COVID-19 through our partners Health Information Research Unit at McMaster University; they also use Application Process Interface (API) from ECRI Clinical Guidelines [
      ECRI
      ECRI clinical guidelines.
      ], International Database of GRADE Guidelines [
      BIGG
      International database of GRADE guidelines (BIGG).
      ], National Institute for Health and Care Excellence, the World Health Organization, the Center for Disease Control, and Guidelines International Network's library. This search is updated twice per week. In parallel we search websites of guideline organizations: European CDC, Public Health Agency of Canada, Scottish Intercollegiate Guidelines, COVID Network Meta-Analysis Initiative and Canadian Task Force on Preventive Health Care. The captured guidelines are then screened manually to ensure that their methodology follows that of a guideline and that their population is COVID-19 suspected, confirmed or at high risk. We do not limit our searches for language or to any COVID-19 topic. Two trained members of the research team use AGREE II instrument to critically appraise the included guidelines. On our map we only report the results of three domains that address credibility: scope and purpose (domain 1), rigor of development (domain 3) and editorial independence (domain 6). Meanwhile, the scores for the other AGREE II domains are kept in our records and provided upon request. When updates of guidelines are published, our team appraises the latest version and modifies the content on the map, accordingly.
      Fig 1
      Fig. 1COVID-19 living map of recommendations and gateway to contextualization process flow diagram.
      To support this complex and intensive workflow for a living recommendation map a customized GRADEpro module (www.gradepro.org) was developed by our technology partner EvidencePrime, Inc. The customization included a guideline level data extraction module, including but not limited to source and search date; and a recommendation level module to capture granular information to be displayed on the map or used for searching the repository. This modular structuring of the information facilitates updating content of the map and will support future guideline development work downstream the recmap (for example, adolopment).
      The unit of organization of the map is the individual recommendation. The user can find what they are looking for through free text searching, browsing a list of recommendations, or consulting a grid that maps recommendations according to topic area (and interventions included) in the columns, and population in the rows. When users select a recommendation of their interest, the detailed information on this individual recommendation is given, including the AGREE-II appraisal, the certainty of the evidence, strength of the recommendation and additional information, depending on what the guideline developers included (for example, evidence profiles).
      A novel feature of this map is the self-directed contextualization of the recommendations using the adolopment module. The GRADE-Adolopment approach [
      • Darzi A.
      • Harfouche M.
      • Arayssi T.
      • Alemadi S.
      • Alnaqbi K.A.
      • Badsha H.
      • et al.
      Adaptation of the 2015 American College of Rheumatology treatment guideline for rheumatoid arthritis for the Eastern Mediterranean Region: an exemplar of the GRADE Adolopment.
      ] uses the EtDs to adopt, adapt or synthesize de novo recommendations to answer context specific questions. Through our map, stakeholders can access the EtDs developed by the guideline developers, and adapt them for their context through the “adolopment module”: a direct linkage to the original guideline project on GRADEpro to access original content, select what needs to be contextualized and go through the steps of development of adapted recommendation. The adapted recommendation can then be pushed to the map as well, and those seeking adolopment can identify relevant high-quality evidence through the linkages to the NIPH and L.OVE platforms of living evidence.
      Dealing with the large volume of guidelines in the times of the pandemic is burdensome, and a prioritization process is necessary to maximize efficiency and responsiveness to changing needs. Our prioritization approach aims at balancing globally trending topics (e.g., masks, vaccination, schools) or the direct requests from stakeholders (e.g., WHO, school administrations) and our team's capacity to process a high but finite number of guidelines at the same time. Our prioritization approach is informed mainly by systematic reviews of guideline prioritization methods and input from experts, and includes three phases: pre-prioritization, prioritization and post-prioritization. The full description of this process is being finalized and will be disseminated in the coming months.
      The overall project is collective effort made possible by a large international and multidisciplinary team of global experts in guideline development, software development, and stakeholders who have joined efforts to develop, maintain and disseminate this easy-to-navigate and freely accessible electronic map. We have organized ourselves in nine working groups: information technology, information science, quality appraisal, data extraction, equity, adolopment, implementation and dissemination, cost/cost-effectiveness and WHO Essential Medicines List. An all-round open steering group meets weekly to review the progress of the work, identify and address upcoming new needs, prioritize topics, and collectively work on reports like this one.
      The engagement of stakeholders is crucial for the uptake, maintenance and sustainability of our map, similarly to their engagement in health research [
      • Concannon T.W.
      • Grant S.
      • Welch V.
      • Petkovic J.
      • Selby J.
      • Crowe S.
      • et al.
      Practical guidance for involving stakeholders in health research.
      ,
      • Domecq J.P.
      • Prutsky G.
      • Elraiyah T.
      • Wang Z.
      • Nabhan M.
      • Shippee N.
      • et al.
      Patient engagement in research: a systematic review.
      . We initially engaged consumer experts in parallel to inform the map's development, and developed a feedback survey that is available on our map for any user to complete. Additionally, we have recently collaborated with Guidelines International Network (GIN) PUBLIC [
      GIN. GIN PUBLIC
      Working groups/GIN PUBLIC.
      ] public versions of guideline recommendations that appear on the our map. Our implementation and dissemination working group is currently working on a plan to optimize the public versions development and dissemination. We have established a mechanism of informing guideline developers that their work will be hosted on our map by direct contact where we introduce them to our work and inform them that their guideline is captured and undergoing appraisal and extraction. They also have the option to provide input or clarification, where needed. Additionally, we have presented webinars on different platforms (GIN, COVID_END, WHO Evidence informed Policy Network, Evidence in Health and CanCOVID), and our team members are available to provide more.
      The COVID19 and the WHO Tuberculosis recommendation maps can save time, resources and ensure uptake of up-to-date recommendations by decision-makers, guideline developers and the public of evidence-informed recommendations through this one stop shop portal. We are currently undertaking a randomized trial to test users’ preferences between the traditional publication of WHO TB guidelines on their website and on the eTB map. We will use the results related to the functionality of the map to improve both maps.
      We do believe that the concept of recommendation mapping is a novel one, and the recent uptake by agencies such as the Canadian Agency for Drugs and Technology in Health (CADTH) suggests others see its value (https://tuberculosis.cadth.ca/). Whether the end-user aim is to identify or to contextualize relevant recommendations, a one stop shop portal of evidence-informed recommendations built with intuitive functionalities, easy to navigate and with a support team ready to guide users across the maps undoubtedly represents a long-needed tool for decision-makers, guideline developers and the public at large.

      Acknowledgments

      Mohamad Tarek Madani, Mark Loeb, Funeka Bango, Megan Thomas, Dina M. Sami Khalifa, Artur Nowak, Lisa Hartling, Joerg J Meerpohl, Marge Reinap, Reem Mustafa, Sarah Scott, Binu Abraham Philip, Verônica Colpani, Tanja Kuchenmüller, Nayê Balzan Schneider, Stephanie Duda, Romina Brignardello-Petersen, Ashley Motilall, Antonio Bognanni, Sura Issa, Sabrina Price, Pablo Alonso-Coello, Ludovic Reveiz, Pema Gurung, Justine Karpusheff, Karen Graham.

      References

        • Gazendam A.
        • et al.
        The “Infodemic” of journal publication associated with the novel coronavirus disease.
        JBJS. 2020; 102: e64
        • Palayew A.
        • et al.
        Pandemic publishing poses a new COVID-19 challenge.
        Nat Hum Behav. 2020; 4: 666-669
        • Furstenau L.B.
        • et al.
        A bibliometric network analysis of coronavirus during the first eight months of COVID-19 in 2020.
        Int J Environ Res Public Health. 2021; 18: 952
        • Zhao S.
        • et al.
        A quality evaluation of guidelines on five different viruses causing public health emergencies of international concern.
        Ann Transl Med. 2020; 8: 500
        • Luo W.-Y.
        • et al.
        Management in the paediatric wards facing novel coronavirus infection: a rapid review of guidelines and consensuses.
        BMJ Open. 2020; 10e039897
        • Dagens A.
        • et al.
        Scope, quality, and inclusivity of clinical guidelines produced early in the COVID-19 pandemic: rapid review.
        BMJ. 2020; 369: m1936
        • NIPH
        Live map of COVID-19 evidence.
        2020 ([cited 2021; Available from:)
        • Epistemonikos
        L.OVE platform..
        2020 ([cited 2021; Available from:)
        • Plus M.
        COVID-19.
        2021 ([cited 2021 15/03/2021]; Available from:)
        • Tricco A.C.
        • Garritty C.M.
        • Boulos L.
        • Lockwood C.
        • Wilson M.
        • McGowan J.
        • et al.
        Rapid review methods more challenging during COVID-19: commentary with a focus on 8 knowledge synthesis steps.
        J Clin Epidemiol. 2020; 126: 177-183
        • Schünemann H.J.
        • Santesso N.
        • Vist G.E.
        • Cuello C.
        • Lotfi T.
        • Flottorp S.
        • et al.
        Using GRADE in situations of emergencies and urgencies: certainty in evidence and recommendations matters during the COVID-19 pandemic, now more than ever and no matter what.
        J Clin Epidemiol. 2020; 127: 202-207
        • Garritty C.
        • Gartlehner G.
        • Nussbaumer-Streit B.
        • King V.J.
        • Hamel C.
        • Kamel C.
        • et al.
        Cochrane rapid reviews methods group offers evidence-informed guidance to conduct rapid reviews.
        J Clin Epidemiol. 2021; 130: 13-22
        • Akl E.A.
        • Morgan R.L.
        • Rooney A.A.
        • Beverly B.
        • Katikireddi S.V.
        • Agarwal A.
        • et al.
        Developing trustworthy recommendations as part of an urgent response (1–2 weeks): a GRADE concept paper.
        J Clin Epidemiol. 2021; 129: 1-11
        • Norris S.L.
        • Sawin V.I.
        • Ferri M.
        • Sastre L.R.
        • Porgo T.V.
        An evaluation of emergency guidelines issued by the World Health Organization in response to four infectious disease outbreaks.
        PLoS One. 2018; 13e0198125
        • Schunemann H.J.
        • Hill S.R.
        • Kakad M.
        • Vist G.E.
        • Bellamy R.
        • Stockman L.
        • et al.
        Transparent development of the WHO rapid advice guidelines.
        PLoS Med. 2007; 4: e119
        • Tugwell P.
        • Knottnerus J.A.
        Adolopment—a new term added to the Clinical Epidemiology Lexicon.
        J Clin Epidemiol. 2017; 81: 1-2
        • Falzon D.
        • Schünemann H.J.
        • Harausz E.
        • González-Angulo L.
        • Lienhardt C.
        • Jaramillo E.
        • et al.
        World Health Organization treatment guidelines for drug-resistant tuberculosis, 2016 update.
        Eur Respir J. 2017; 49
        • Darzi A.
        • Harfouche M.
        • Arayssi T.
        • Alemadi S.
        • Alnaqbi K.A.
        • Badsha H.
        • et al.
        Adaptation of the 2015 American College of Rheumatology treatment guideline for rheumatoid arthritis for the Eastern Mediterranean Region: an exemplar of the GRADE Adolopment.
        Health Qual Life Outcomes. 2017; 15: 183
        • Collaboration T.A.
        Development and validation of an international appraisal instrument for assessing the quality of clinical practice guidelines: the AGREE project.
        Qual Saf Health Care. 2003; 12: 18-23
        • Schunemann H.J.
        • Wiercioch W.
        • Etxeandia I.
        • Falavigna M.
        • Santesso N.
        • Mustafa R.
        • et al.
        Guidelines 2.0: systematic development of a comprehensive checklist for a successful guideline enterprise.
        CMAJ. 2014; 186: E123-E142
        • Brouwers M.C.
        • Kerkvliet K.
        • Spithoff K.
        The AGREE Reporting Checklist: a tool to improve reporting of clinical practice guidelines.
        BMJ. 2016; 352: i1152
        • Chen Y.
        • Yang K.
        • Marušic A.
        • Qaseem A.
        • Meerpohl J.
        • Flottorp S.
        • et al.
        A reporting tool for practice guidelines in health care: the RIGHT statement.
        Ann Intern Med. 2017; 166: 128-132
        • Alia E.
        • Grant-Kels J.M.
        The trustworthiness and transparency in clinical practice guidelines versus the ongoing damaging power of direct and indirect conflict of interest.
        Clin Dermatol. 2020; 38: 254-256
        • Guyatt G
        • A.E. Hirsh J
        • Kearon C
        • Crowther M
        • Gutterman D
        • Lewis SZ
        • et al.
        The vexing problem of guidelines and conflict of interest: a potential solution.
        Ann Intern Med. 2010; 152: 738-741
        • Schunemann H.J.
        • Al-Ansary L.A.
        • Forland F.
        • Kersten S.
        • Komulainen J.
        • Kopp I.B.
        • et al.
        Guidelines international network: principles for disclosure of interests and management of conflicts in guidelines.
        Ann Intern Med. 2015; 163: 548-553
        • Schunemann H.J.
        • Osborne M.
        • Moss J.
        • Manthous C.
        • Wagner G.
        • Sicillian L.
        • et al.
        An Official American Thoracic Society policy statement: managing conflict of interest in professional societies.
        Am J Respir Crit Care Med. 2009; 180: 564-580
        • Boyd E.A.
        • Bero L.A.
        Improving the use of research evidence in guideline development: managing conflicts of interests.
        Health Res Policy Syst. 2006; 4: 16
        • Amir Qaseem F.F.
        • Macbeth F
        • Ollenschläger G
        • Phillips S
        Philip van der Wees, for the Board of Trustees of the Guidelines International Network*, Guidelines International Network: Toward International Standards for Clinical Practice Guidelines.
        Ann Intern Med. 2012; 156: 525-531
        • Hajizadeh A
        • L.T. Dennis Falzon F
        • Mertz D
        • Nieuwlaat R
        • Gebreselassie N
        • Jaramillo E
        • et al.
        Recommendation mapping of the World Health Organization's guidelines on tuberculosis: a new approach to digitizing and presenting recommendations.
        J Clin Epidemiol. 2021; (In press)
        • ECRI
        ECRI clinical guidelines.
        2020
        • BIGG
        International database of GRADE guidelines (BIGG).
        2020
        • Concannon T.W.
        • Grant S.
        • Welch V.
        • Petkovic J.
        • Selby J.
        • Crowe S.
        • et al.
        Practical guidance for involving stakeholders in health research.
        J Gen Intern Med. 2019; 34: 458-463
        • Domecq J.P.
        • Prutsky G.
        • Elraiyah T.
        • Wang Z.
        • Nabhan M.
        • Shippee N.
        • et al.
        Patient engagement in research: a systematic review.
        BMC Health Serv Res. 2014; 14: 89
        • GIN. GIN PUBLIC
        Working groups/GIN PUBLIC.
        Guidelines International Network, 2020 (Available from:)
        • Brouwers M.C.
        • Kho M.E.
        • Browman G.P.
        • Burgers J.S.
        • Cluzeau F.
        • Feder G.
        • et al.
        AGREE II: advancing guideline development, reporting and evaluation in health care.
        CMAJ. 2010; 182: E839-E842