If you don't remember your password, you can reset it by entering your email address and clicking the Reset Password button. You will then receive an email that contains a secure link for resetting your password
If the address matches a valid account an email will be sent to __email__ with instructions for resetting your password
Corresponding author. Department of Clinical Epidemiology and Biostatistics, McMaster University Health Sciences Centre, Room 2C16, 1280 Main Street West, Hamilton, Ontario L8N 4K1, Canada. Tel.: +1-905 525 9140x24931; fax: +1-905 522 9507.
Department of Clinical Epidemiology and Biostatistics, McMaster University, 1280 Main Street West, Hamilton, Ontario L8N 4K1, CanadaDepartment of Medicine, McMaster University, 1280 Main Street West, Hamilton, Ontario L8N 4K1, Canada
Department of Clinical Epidemiology and Biostatistics, McMaster University, 1280 Main Street West, Hamilton, Ontario L8N 4K1, CanadaDepartment of Medicine, McMaster University, 1280 Main Street West, Hamilton, Ontario L8N 4K1, Canada
Department of Clinical Epidemiology and Biostatistics, McMaster University, 1280 Main Street West, Hamilton, Ontario L8N 4K1, CanadaDepartment of Internal Medicine/Nephrology, University of Missouri-Kansas City, 2301 Holmes Street, Kansas City, MO 64108, USADepartment of Biomedical and Health Informatics, University of Missouri-Kansas City, 2411 Holmes Street, Kansas City, MO 64108, USA
Division of Cardiology, Department of Medicine, Veterans Affairs Medical Center, 3495 Bailey Avenue, Buffalo, NY 14215, USADepartment of Internal Medicine, University at Buffalo, The State University of New York, 3435 Main street, Buffalo, NY 14214, USA
Institute of Health Policy, Management and Evaluation, University of Toronto, 155 College St, Toronto, ON M5T 3M6, CanadaEvidence-Based Dentistry Unit, Faculty of Dentistry, Universidad de Chile, Sergio Livingstone Pohlhammer 943, Independencia, Santiago 8380492, Chile
Department of Clinical Epidemiology and Biostatistics, McMaster University, 1280 Main Street West, Hamilton, Ontario L8N 4K1, CanadaDepartment of Internal Medicine, Pontificia Universidad Católica de Chile, Alameda 340, Santiago 8331150, Chile
Hospital Moinhos de Vento, Rua Ramiro Barcelos 910, Bairro Moinhos de Vento, Porto Alegre, BrazilNational Institute of Science and Technology for Health Technology Assessment, Federal University of Rio Grande do Sul, Rua Ramiro Barcelos, No. 2359, 90035-903 Porto Alegre, RS, Brazil
Department of Clinical Epidemiology and Biostatistics, McMaster University, 1280 Main Street West, Hamilton, Ontario L8N 4K1, CanadaDepartment of Medicine, McMaster University, 1280 Main Street West, Hamilton, Ontario L8N 4K1, Canada
Cochrane Germany, Medical Center—University of Freiburg, Breisacher Strasse 153, Freiburg 79110, GermanyCentre de Recherche Épidémiologie et Statistique Sorbonne Paris Cité—U1153, Inserm/Université Paris Descartes, Cochrane France, Hôpital Hôtel-Dieu, 1 place du Parvis Notre Dame, 75181 Paris Cedex 04, France
Department of Clinical Epidemiology and Biostatistics, Pontificia Universidad Javeriana, Hospital Universitario de San Ignacio, Cr. 7 40-62 2nd Floor, Bogotá, Colombia
Department of Clinical Epidemiology and Biostatistics, McMaster University, 1280 Main Street West, Hamilton, Ontario L8N 4K1, CanadaDepartment of Internal Medicine, Pontificia Universidad Católica de Chile, Alameda 340, Santiago 8331150, Chile
Department of Clinical Epidemiology and Biostatistics, McMaster University, 1280 Main Street West, Hamilton, Ontario L8N 4K1, CanadaDepartment of Family Medicine, McMaster University, David Braley Health Sciences Centre, 100 Main Street West, 6th Floor, Hamilton, Ontario L8P 1H6, Canada
Department of Clinical Epidemiology and Biostatistics, McMaster University, 1280 Main Street West, Hamilton, Ontario L8N 4K1, CanadaDepartment of Family Medicine, McMaster University, David Braley Health Sciences Centre, 100 Main Street West, 6th Floor, Hamilton, Ontario L8P 1H6, Canada
Department of Clinical Epidemiology and Biostatistics, McMaster University, 1280 Main Street West, Hamilton, Ontario L8N 4K1, CanadaTecnologico de Monterrey School of Medicine, Avda. Morones Prieto 3000 pte. CP 64710 Monterrey, Nuevo León, Mexico
Department of Internal Medicine, Faculty of Health Sciences, American University of Beirut, P.O. Box: 11-0236, Riad-El-Solh Beirut 1107 2020, Beirut, LebanonThe Kirby Institute, University of New South Wales, Wallace Wurth Building, University of New South Wales, Sydney, New South Wales 2052, Australia
Department of Clinical Epidemiology and Biostatistics, McMaster University, 1280 Main Street West, Hamilton, Ontario L8N 4K1, CanadaDepartment of Internal Medicine, Faculty of Health Sciences, American University of Beirut, P.O. Box: 11-0236, Riad-El-Solh Beirut 1107 2020, Beirut, Lebanon
Guideline developers can: (1) adopt existing recommendations from others; (2) adapt existing recommendations to their own context; or (3) create recommendations de novo. Monetary and nonmonetary resources, credibility, maximization of uptake, as well as logical arguments should guide the choice of the approach and processes.
Objectives
To describe a potentially efficient model for guideline production based on adoption, adaptation, and/or de novo development of recommendations utilizing the Grading of Recommendations Assessment, Development and Evaluation (GRADE) Evidence to Decision (EtD) frameworks.
Study Design and Setting
We applied the model in a new national guideline program producing 22 practice guidelines. We searched for relevant evidence that informs the direction and strength of a recommendation. We then produced GRADE EtDs for guideline panels to develop recommendations.
Results
We produced a total of 80 EtD frameworks in approximately 4 months and 146 EtDs in approximately 6 months in two waves. Use of the EtD frameworks allowed panel members understand judgments of others about the criteria that bear on guideline recommendations and then make their own judgments about those criteria in a systematic approach.
Conclusion
The “GRADE-ADOLOPMENT” approach to guideline production combines adoption, adaptation, and, as needed, de novo development of recommendations. If developers of guidelines follow EtD criteria more widely and make their work publically available, this approach should prove even more useful.
We introduce a methodology that combines the advantages of adoption, adaptation, and de novo development of recommendations (“GRADE-ADOLOPMENT”) based on the Grading of Recommendations Assessment, Development and Evaluation (GRADE) Evidence to Decision (EtD) frameworks. We tested the methodology in 22 guidelines.
What this adds to what was known?
•
The structure of the GRADE EtD frameworks and the criteria that determine the direction and strength of a recommendation allows adolopers to create recommendations appropriate for their context.
What is the implication and what should change now?
•
By using the EtD criteria to transparently present the research evidence and the associated judgments, guideline producers will facilitate ADOLOPMENT of their recommendations by others.
1. Introduction
The preeminent role of health guidelines is to assist with evidence-based decision-making for individuals, populations, and systems in health care [
]. Although many organizations develop guidelines to provide advice on an international level, there often are legal reasons, regulatory requirements, or perceived needs to produce guidelines on a national or regional level. Perceived needs may originate in the justified belief that guidelines must be developed in the context they are used in. However, some organizations tasked with producing guidelines may lack the monetary and nonmonetary resources to produce evidence-based guidelines independently. These guideline developers typically have three choices: (1) adopt existing recommendations as they are; (2) adapt existing recommendations to their own context; or (3) develop recommendations de novo based on available evidence syntheses. Although all of these approaches should start with identifying appropriate guideline panels, the approaches differ importantly with regard to the required investments.
Adoption of guidelines means the use of an existing, trustworthy recommendation without modification of the original recommendation and providing information on how to implement it. Trustworthy recommendations are those that follow best standards or practices for guideline development. It begins with guideline panels reviewing guidelines and ends with agreeing with the judgments that determine the direction and strength of recommendations made by the original guideline developer. In the ideal case, this should be based on review and agreement with the methods of development and judgments that influenced the original recommendation. The adopted recommendation would have the same specific population, intervention, and comparators as the original recommendation and the same certainty in the evidence rating. However, the choice of the guideline scope and the individual recommendations follows from their availability. Yet, it is the cheapest and quickest way of developing a guideline.
As for adoption, adaptation involves identifying the pertinent health care questions, searching for existing guidelines that addressed those questions, critically appraising them, and deciding whether to accept or modify all or selected recommendations. This decision also requires considering whether recommendations are credible, up to date, acceptable, applicable, and feasible to implement given the cultural and organizational context. The adapted recommendation may have a change in the specific population, intervention, comparator than the original recommendation and a different certainty in the evidence. The adapted recommendation will provide additional information on “conditions,” monitoring, implementation, and implications for research.
Although adaptation and adoption should focus on issues that are relevant for the health care setting, both processes are often driven and initiated by the availability of guidelines. Adoption and adaptation serve two primary purposes: (1) using limited resources more efficiently by building on existing efforts to provide local, regional, or national guidance; and (2) considering factors that are specific to these settings to enhance usability for the intended target groups. Using this approach, guideline developers must choose which recommendations to adapt. Advice given to the World Health Organization (WHO) in 2005 suggested criteria to select recommendations in guidelines that require adaptation, such as variation in values or cost across settings [
]. In addition, some approaches like ADAPTE provide detailed guidance for potentially modifying guidelines produced in one setting for use in a different setting [
]. Although adaptation of existing guidelines is thought to reduce work required to produce guidelines, the approach becomes resource intensive if information that is required for adaptation is not available. Furthermore, some international organizations develop guidelines that are intended to have wide applicability to support adoption or adaptation [
]. For example, WHO produces guidelines that may focus on low- and middle-income settings. These guidelines may require additional consideration or adaptation of contextual issues that may not be fully known or suspected to vary across settings, during centralized guideline processes [
Transparently laying out the judgments that a guideline panel makes when formulating recommendations would facilitate their later adaptation. However, existing guidelines often do not provide the necessary details about this process and other decisions necessary to work on their adaptation and adoption [
]. Unfortunately, this makes de novo recommendation development often unavoidable because evidence syntheses are not appropriately developed or do not cover all criteria that are relevant for local decision-making [
]. Thus, proper adoption or adaptation of recommendations requires transparent description of the processes used by the original guidelines, including the methodology used and how conflicts of interest were managed.
Development of de novo recommendations, on the other hand, involves formulating new questions and seeking to answer them in guidelines that contain recommendations not included in original guidelines [
]. This approach can be based on existing evidence synthesis such as systematic reviews or health technology assessments (HTAs) that the guideline developer identifies as relevant for their questions. Original guidelines may still play a role in de novo development by making evidence syntheses available that may lead to recommendations that the original guideline developer did not consider. It should follow good practice to produce trustworthy guidelines described by several influential groups [
]. Features of trustworthy recommendations include the conduct of systematic reviews and transparent descriptions of the underlying certainty in the evidence and how guideline developers move from evidence to recommendations [
However, considerations that guide the choice of the guideline development approach include the availability of monetary and nonmonetary resources, credibility, maximization of uptake, the benefits of sharing information widely, and the avoidance of duplication of efforts. Organizations that produce guidelines will need to decide on the best approaches to develop guidelines and to design detailed strategies and build capacity to implement them [
]. Previous work with international organizations and health authorities on guideline development has addressed the need to compile and update evidence in sharable formats while allowing for consideration of context-specific factors [
GRADE Evidence to Decision (EtD) frameworks: a systematic and transparent approach to making well informed healthcare choices. 2: Clinical practice guidelines.
GRADE Evidence to Decision (EtD) frameworks: a systematic and transparent approach to making well informed healthcare choices. 2: Clinical practice guidelines.
]. The EtD frameworks provide information on criteria that bear on guideline recommendations (e.g., health benefits, harms, certainty in the best available evidence, cost, feasibility) and how the panellists judge the effect of this information on the final recommendation. The EtD frameworks may facilitate the adoption or adaptation of guidelines to the setting, context, and culture of a specific jurisdiction or country.
We developed and tested an approach for adoption, adaptation, and de novo guideline development based on the GRADE EtD frameworks. To complete this work, we applied prior work on adaptation of guidelines to address the challenges guideline developers face [
]. The main objective of this article is to describe this approach based on applying elements of it to 22 guidelines as part of a new national guideline program by the Ministry of Health in Saudi Arabia. We call this approach “GRADE-ADOLOPMENT” of guidelines, expressing the combined use of adoption, adaptation, and de novo recommendations to provide trustworthy guidelines.
2. Methods
2.1 General organization and planning
We developed GRADE-ADOLPMENT as a result of establishing a new national guideline program by the Ministry of Health in the Kingdom of Saudi Arabia (KSA). Our work began by creating a handbook for guideline production that described the approach and built on our prior work [
]. The project planning began in June 2012 and implementation of the guideline development started in July 2013. “Wave 1” included generating practice guidelines on 10 different topics in 2013, and “wave 2” included generating 12 practice guidelines from 2014 to 2015 [
]. Our main goal was to guide the development of recommendations for questions that were relevant for stakeholders in the KSA, using the GRADE EtDs tables.
2.2 Groups and roles
Methodologists from the Department of Clinical Epidemiology and Biostatistics at McMaster University, specifically the McMaster GRADE Center (cebgrade.mcmaster.ca), led the methodological and guideline development work (McMaster group). These trained guideline methodologists supervised the development of each guideline. We invited additional methodologists for wave 2 of the project to support development of the guidelines. Methodologists were responsible for communicating with guideline panels, conducting literature searches, updating systematic reviews, developing draft EtDs, and chairing guideline panel meetings. The Saudi Center for Evidence-Based Health Care (EBHC) was responsible for selection of panel members, final agreement of guideline topics, communication, and logistics. Each guideline panel included 5 to 10 KSA expert members from multidisciplinary backgrounds, including some patient representatives.
Guideline panel members were involved in the prioritization of individual health care questions for each topic, informing the review of evidence specific for the KSA setting, formulation of recommendations during guideline panel meeting, and drafting a guideline manuscript for peer-reviewed publication. The McMaster group created a training package for panel members. The package included a narrated online presentation of a summary of the guideline development process and the GRADE approach and online videos on how to interpret GRADE evidence tables [
]. The training package also included an overview of the EtD framework (available on http://cebgrade.mcmaster.ca/ksaproject/). Conflicts of interest were declared and managed according to rules described in the Saudi Arabian Guideline Handbook, largely based on the WHO approach [
Moving from evidence to developing recommendations in guidelines: article 11 in Integrating and coordinating efforts in COPD guideline development. An official ATS/ERS workshop report.
For the first wave of guidelines, the McMaster group proposed potential topics for consideration by the Ministry of Health, of which the Ministry representatives selected 10. For wave 2, the EBHC solicited topics of interest from Ministry of Health stakeholders during a detailed priority-setting exercise (Appendix 1 at www.jclinepi.com), which were then assessed by the methodologists for feasibility for guideline development. Feasibility for wave 1 required the existence of published guidelines that used the GRADE approach and had publically available evidence summaries in the form of GRADE Summary of Findings (SoFs) tables or evidence profiles (EPs), as well as existing systematic reviews with access to original search strategies for the questions of interest. Ministry of Health representatives selected 12 topics in wave 2.
2.4 Prioritizing questions for selected guidelines
For each selected guideline, we used a formal process to prioritize approximately 3–10 key clinical questions for inclusion during wave 1 and 10–15 questions during wave 2, based on the questions addressed in existing evidence syntheses. Guideline panel members completed online surveys to rate the relative importance of clinical questions for the Saudi Arabia health care setting. We used a 9-point Likert scale (1-least important; 9-most important). Panelists were asked to consider the patient's perspective, the availability of the interventions, and legal issues (e.g., intervention not available in KSA), but not to exclude questions for resource considerations (e.g., potential financial barriers for implementation of the proposed interventions). Mean and median importance ratings of questions guided inclusion in the guideline. To ensure that guidelines comprehensively addressed the topic with a complete set of recommendations, questions deemed complementary to those rated as important (e.g., questions that together addressed a complete diagnostic strategy) were also included. The selected questions were sent to panelists for approval, with opportunity for further input before finalization.
2.5 Using the GRADE Evidence to Decision frameworks
Our goal was to complete GRADE EtDs for each guideline recommendation as a central element of the guideline development (see EtD in Appendix 2 at www.jclinepi.com). The EtDs included the summary of evidence about the benefits and harms of the intervention option(s) being considered, but also any information found about the importance of the problem (e.g., baseline risk), patients' values and preferences, resource use and costs, feasibility, acceptability, and potential impact on health equity of recommending specific intervention options in the context of the KSA health care setting and affected stakeholders (see Table 1 for a description of the criteria of the EtD). For both waves and for specific priority topics and clinical questions, we searched for and selected existing highly credible guidelines, evidence syntheses, including systematic reviews and HTAs. When needed, we updated these evidence syntheses on intervention effects and then conducted supplementary searches for evidence to complete the EtD frameworks specific to the local health care setting and formulating recommendations by considering the key context-specific factors.
Table 1Criteria that influence the strength and direction in the Evidence to Decision frameworks
Criteria
How the factor influences the direction and strength of a recommendation
Problem
The problem is determined by the importance and frequency of the health care issue that is addressed (burden of disease, prevalence, or baseline risk). If the problem is of great importance, a strong recommendation is more likely.
Values and preferences
This describes how important health outcomes are to those affected, how variable they are and if there is uncertainty about this. Values and preferences or the importance of outcomes
Certainty in the evidence
The higher the certainty in the evidence, the more likely is a strong recommendation.
Health benefits and harms and burden and their balance
This requires an evaluation of the absolute effects of both the benefits and harms and their importance. The greater the net benefit or net harm, the more likely is a strong recommendation for or against the option.
Resource implications
This describes how resource intense an option is, if it is cost-effective and if there is incremental benefit. The more advantageous or clearly disadvantageous these resource implications are the more likely is a strong recommendation.
Equity
The greater the likelihood to reduce inequities or increase equity and the more accessible an option is, the more likely is a strong recommendation.
Acceptability
The greater the acceptability of an option to all or most stakeholders, the more likely is a strong recommendation.
Feasibility
The greater the acceptability of an option to all or most stakeholders, the more likely is a strong recommendation.
2.6 Updating systematic reviews of health effects and identifying local data
The methodologists updated systematic reviews of health outcomes of interventions, if the source systematic reviews were older than 3 months using the original search strategies from the existing systematic reviews following standard systematic review methods. The methodologists also conducted rapid systematic reviews to identify studies on patients' values and preferences and economic analyses relevant to the KSA health care setting (e.g., values and preferences in the Middle East region) with the help of research librarians. Additionally, we solicited input from panel members about local studies and information on patients' values and preferences, cost-effectiveness, resource use as well as population prevalence and incidence of disease as applicable to the local health care setting.
2.7 Preparing GRADE evidence tables and Evidence to Decision frameworks
For each guideline question, the methodologists summarized the evidence in new evidence tables: GRADE SoF tables or EPs [
Chapter 11: presenting results and ‘Summary of findings' tables.
in: Higgins J.P.T. Green S. Cochrane Handbook for Systematic Reviews of Interventions Version 510. The Cochrane Collaboration,
West Sussex, UK2011: 2011
]. The evidence tables summarized the relative effects of alternative management strategies on the outcomes of interest, the certainty in the available evidence for each outcome, and the judgments that bear on the certainty rating. Each evidence table was then peer reviewed by one of two senior methodologists (H.J.S. and J.B.). The methodologists then completed draft EtD frameworks for each guideline question (see Appendix 2 at www.jclinepi.com for an EtD example). The EtD frameworks facilitated and structured panels' discussions about baseline risk, patients' values and preferences, resource use and cost, health equity, feasibility, and acceptability for the intervention options being considered. We used the GRADEpro app (www.gradepro.org) to produce evidence tables and EtD frameworks [
2.8 Formulating and rating strength of recommendations
For each wave and guideline, a 2-day panel meeting was held including an introduction to guideline development. One methodologist chaired the guideline panel meeting and was supported by one or two other methodologists. Panels reviewed the evidence tables and the EtD tables and formulated recommendations through consensus or voting, if necessary. The EtD frameworks were used to record panels' judgments when considering the criteria to determine the direction and strength of a recommendation. The conclusions included considerations for implementation, monitoring, evaluation of the recommendations as well as local research needs.
2.9 Arriving at a final framework for GRADE-ADOLOPMENT
Based on the experience with the recommendations we developed as part of this guideline production effort, we arrived at a final framework for GRADE-ADOLOPMENT. We considered the planned approach based which of the planned and implemented steps required modification to be practical and trustworthy. We separated the three key issues of adoption, adaptation, and de novo creation. We achieved this by reviewing examples of the final recommendations and using them as examples for the final framework.
3. Results
3.1 Completed guidelines
The effective time needed to complete 10 guidelines with 80 recommendations in wave 1 was approximately 4 months, and to complete 12 guidelines with 146 recommendations in wave 2, it was approximately 6 months (Table 2). The guideline topics were broad, including topics for primary care, specialist care, and population screening and covering areas of hematology, cardiology, neurology, nephrology, maternal-fetal medicine, allergy, cancer, and others.
Table 2Clinical practice guideline topic areas for phases 1 and 2 of KSA guideline project
Clinical practice guideline
Guideline topic
Number of recommendations
Wave 1
1
Antithrombotic treatment of patients with nonvalvular atrial fibrillation
10
2
Treatment of venous thromboembolism
8
3
Use of thrombolytic therapy in acute stroke
6
4
Prevention of venous thromboembolism in patients with stroke
8
5
Diagnosis of suspected first lower extremity deep vein thrombosis
24
6
Use of screening strategies for detection of breast cancer
5
7
Screening and treatment of precancerous lesions for cervical cancer prevention
6
8
Allergic rhinitis
8
9
Role of vitamin D, calcium, and exercise in fracture prevention in elderly
4
10
Timing of initiation of dialysis
1
Wave 2
1
Management of ST-elevation myocardial infarction
11
2
Prevention of VTE in surgical patients
18
3
Prevention of VTE in nonsurgical patients
20
4
Management of eclampsia
8
5
Management of breast lump and primary breast cancer
The conceptual approach to GRADE-ADOLOPMENT that resulted from adopting, adapting, or developing 226 recommendations in 22 guidelines is described in Fig. 1 and in more detail in Appendix 3 at www.jclinepi.com. It is the result of developing a detailed approach to guideline development for a new national guideline program and is informed by prior work on adaptation, work on the EtD frameworks, and practical experience gathered in this large guideline development project (P. Alonso-Coello et al., unpublished data.) [
]. The differences between the framework we present here and the original intended approach for the KSA guideline program were small and driven by practical reasons. For example, there was a lack of relevant information from the original guidelines about the EtD criteria in most situations. In addition, we often used existing evidence syntheses rather than completed guidelines and recommendations in wave 2. Thus, information that was required to complete the EtD frameworks for the KSA guidelines was incomplete for most recommendations. The vast majority recommendations in the 22 guidelines recommendations were, therefore, adapted or developed de novo. However, adoption of some recommendations was facilitated by availability of information and judgments on the EtD criteria. The cornerstones of the ADOLOPMENT approach (Fig. 1 and Appendix 3 at www.jclinepi.com) are to:
1.
Identify and prioritize credible existing guidelines or evidence syntheses of interest and relevance. This step should involve the relevant stakeholders and proper priority setting (see steps 2 through 5 in Appendix 1 at www.jclinepi.com).
2.
Evaluate and complete GRADE EtD Frameworks for each recommendation. This step involves identifying and reviewing information of existing EtD frameworks or identifying information that informs the EtD criteria and completing a new EtD for the adoloped recommendation.
3.
Final adoption, adaptation, or de novo creation of recommendations based on the extent of changes made to the original recommendation or degree of work involved.
Fig. 1GRADE-ADOLOPMENT of guideline recommendations. Brief description of GRADE-ADOLOPMENT. A more detailed description of the steps is shown in Appendix 3 at www.jclinepi.com. Guideline topics are identified by evaluating credible existing guidelines or evidence syntheses after or before priorities are set by a guideline group. This process should involve the relevant stakeholders. It involves deciding to accept or modify whole guidelines or their specific recommendations by considering whether they are credible, up to date, acceptable, and applicable given the cultural and organizational context. The next critical step after identification of possibly matching recommendations includes completing or utilizing GRADE Evidence to Decision (EtD) frameworks for recommendations for either a matched recommendation or a new recommendation. It will often require conducting updates of existing systematic reviews. To identify major and minor updates or define a new systematic review, the criteria in Appendix 4 at www.jclinepi.com and the work by Garner et al. are useful
. Depending on agreement with the information presented in the existing guidelines or requirements for new evidence, recommendations are adopted or adapted. If no information or recommendation is available, a new recommendation is developed. GRADE, Grading of Recommendations Assessment, Development and Evaluation.
We will describe adoption, adaptation, or de novo creation in the following sections based on practical examples.
3.3 Adoption of recommendations
The GRADE EtDs ask guideline panels to consider criteria that influence the direction and strength of a recommendation as well as its implementation. The guideline panel should evaluate the evidence, judgments, and decisions of the original recommendation from the source guideline and address agreement and disagreement with these judgments. If judgments do not differ sufficiently to change the direction and strength of a recommendation, panel members will adopt the recommendation as is. If judgments differ, panel members will want to change, i.e., adapt, the recommendation. Whether they adopt or adapt the recommendation, the EtD framework helps considering criteria that address implementation and possible research gaps, even those specific to the setting. The EtD framework in one of the Ministry of Health of KSA guideline [
] shows the an adopted recommendation as “The KSA MoH guideline panel recommends against ‘intent-to-start-early’ rather than ‘intent-to-defer’ strategy for initiating dialysis in adult patient (age 18 years or more) with stage 5 (a glomerular filtration rate <15 mL/min/1.73 m2) (strong recommendation, moderate quality of evidence).” This recommendation was adopted after considering all criteria in the EtD framework, local evidence, and additional considerations in the EtD. Adoption was facilitated by understanding the judgments that led to formulating the original recommendation.
3.4 Adaptation of recommendations
A guideline panel following the EtD framework may decide that their judgments differ from those of the original guideline panel and, thus, they may provide a recommendations that differs from the original one. For example, although the Canadian Task Force guideline on breast cancer screening provided a weak recommendation against screening in 40- to 50-year old women, the KSA guideline panel made a conditional recommendation in favor of screening in this age group because of the presumed higher baseline risk (affecting the problem criterion in the EtD and the absolute risk reduction in the benefits and harms criterion) in younger women in the KSA. Constructing the EtD framework and extracting information from the original recommendation allowed understanding and explaining the reasons for disagreement, in this case presumed different baseline risks.
3.5 De novo development
A recommendation that addressed the question “Should multivessel vs. culprit vessel only percutaneous coronary interventions be used in patients with acute ST-wave elevation myocardial infarction and multivessel coronary artery disease be used” identified two new trials compared with an evidence synthesis used for a United Kingdom National Institute for Health and Care Excellence (NICE) guideline. The number of trial participants increased from approximately 200 (two trials) to 1,000 (four trials) (Appendix 2 at www.jclinepi.com). Although the NICE guideline panel refrained from developing a recommendation because of paucity of the evidence, the KSA panel, through updating the search, developed a new recommendation. Although the balance based on the clinical evidence for effects favored benefits over harms for patients, consideration of factors such as the local baseline risk and feasibility of administering that intervention in the local health care setting impacted on the direction and the strength (i.e., weak/conditional or strong recommendation) of the recommendation (The panel suggests multivessel PPCI over culprit-only PCI for patients with multivessel coronary artery disease undergoing PPCI [conditional recommendation; low-quality evidence]).
3.6 Required resources
Although the time and resources required were less than that of developing all guidelines de novo, the approach still required specific expertise in guideline development and evidence synthesis, having a designated methodology lead for each guideline, research librarian support for updating literature searches, methodological expertise for updating evidence syntheses and analyses, and experience in facilitation of panel meetings.
4. Discussion
We established an approach that we call “GRADE-ADOLOPMENT” of guideline recommendations. It combines advantages of adoption, adaptation, and de novo guideline development. The approach builds on the GRADE EtD framework and earlier suggestions offered to the WHO to allow formulation of recommendations for a specific health care setting (P. Alonso-Coello et al., unpublished data.) [
Our approach has a number of strengths. Utilizing existing evidence syntheses, in particular those used in guidelines, as a starting point for guideline ADOLOPMENT avoided conducting full systematic reviews about health effects for many questions, a major resource requirement for guidelines, and is in line with visions for better guideline development [
]. We completed ADOLOPMENT of recommendations in a relatively short time frame, less than 1 year of effective time spent for each of two waves with a large number of guidelines and recommendations. This is considerable shorter than described by major guideline developers with time estimates of up to 3 years for single guidelines [
]. The approach allowed inclusion of panel members for capacity building, while achieving locally contextualized guidance and recommendations for health care providers and stakeholders in the national setting. Additional strengths of the process include, use of the EtD framework which allowed transparent recording of the panels' decisions and considerations made in reaching a recommendation. This transparency allows local users of the guideline to assess the panel's decision-making process, likely enhancing acceptability and credibility of the guidelines and recommendations, but also allows for updating of guidelines when new evidence, especially evidence for the local health care setting becomes available. The guidelines developed addressed a wide range of topic areas, and with the use of the EtDs and software tools (www.gradepro.org), recommendations were formulated specifically for the KSA health care setting. Another advantage of our approach is that the methodology as well as the tools and materials used in the project provide a generalizable approach for various health care and country-specific contexts. It may prove particularly helpful for international organizations such as WHO that often develop global recommendations, which require contextualization and localization. The EtDs facilitate identifying the criteria that may alter the strength or direction of a recommendation.
It is important to note some of the limitations of the described approach. Most of the available evidence syntheses were restricted to effects of intervention, and little evidence was available for the other EtD criteria. Thus, less resourced guideline developers may struggle to complete the information required for the EtDs. However, guideline development projects in general face this challenge, and ADOLOPMENT will, over time, become more efficient when EtDs will be more widespread. Furthermore, the EtDs provide required structure and help identifying gaps of knowledge for those who choose to adopt, adapt, or de novo create recommendations. In wave 1 of this work, we screened available existing evidence syntheses, particularly ones we were familiar with from our previous work on other guidelines. Reliance on existing evidence synthesis may lead to focus on areas that are already well explored. Although representing priority areas, this does not allow priority setting that is entirely panel or health care system driven. Until guideline developers make the criteria clearer that are included in the EtD framework, guideline adolopers will need to extract information from existing guidelines that is often not provided in transparent formats. Thus, the adoption element of GRADE-ADOLOPMENT will be facilitated by the availability of EtD frameworks which, like in this effort, are becoming increasingly available. The GRADE app GRADEpro currently receives an adaptation module that will be further evaluated and refined in subsequent ADOLOPMENT work. The results of the current and future work are available for guideline adolopers on the GRADE database (http://dbep.gradepro.org/).
Many current recommendations are supported by limited evidence about EtD criteria. For example, our extensive searches for patients' values and preferences specific to the local setting as well as cost-effectiveness and resource use were well received by panel members but often produced limited data. We also involved the local experts in helping to identify new information that was relevant to the local setting. ADOLOPMENT requires involvement of local stakeholders and experts throughout the guideline development process to ensure that the questions, evidence, and recommendations are contextualized to address local needs and the health care system structure. The ADOLOPMENT process also facilitates buy-in with decisions and legitimization of the process among those who will eventually disseminate and implement the guidelines.
Compared to approaches to guideline adaptation, such as the ADAPTE process, the methodology we have outlined differs in that panels did not evaluate the acceptability and applicability of recommendations from existing guidelines for their health care setting. Instead, by focusing on questions and using EtDs, the process began with updating existing evidence syntheses, and conducting systematic searches for evidence specific to the criteria of the EtD and the local context, panel members were able to formulate context-specific recommendations. Thus, the approach did not begin with focusing on the evaluation of existing recommendations but the criteria that are used to decide about a recommendation. By facilitating panels to adolop recommendations, we believe the approach enabled greater buy-in and ownership of the guideline produced, which aids in dissemination and uptake in practice and supports guideline capacity building.
Adoloped recommendations often differ from other guidelines that used the same evidence about intervention effects. This underlines the importance of a combination of adoption, adaptation, and de novo development. The reasons for alternative recommendations were transparently described in the EtD (e.g., differences in baseline risk, new evidence about the effects of interventions, different values, and preferences or resource considerations).
5. Conclusion
We used an approach to guideline production that we call GRADE-ADOLOPMENT because it combines the advantages of adoption, adaptation, and de novo development of guidelines. GRADE EtDs are a core component of the approach, as they transparently present setting-specific evidence (P. Alonso-Coello et al., unpublished data.) [
]. The approach facilitates structured interaction and deliberation with panel members during guideline panel meetings and can save important resources.
Acknowledgments
The authors are grateful to McMaster University research librarians Jennifer Lawson, Tamara Navarro, and Rachel Couban for their assistance in developing literature search strategies and to Nourah Al Moufarreh of the Saudi Center for Evidence-Based Heath Care for her help with project management.
GRADE Evidence to Decision (EtD) frameworks: a systematic and transparent approach to making well informed healthcare choices. 2: Clinical practice guidelines.
Moving from evidence to developing recommendations in guidelines: article 11 in Integrating and coordinating efforts in COPD guideline development. An official ATS/ERS workshop report.
Chapter 11: presenting results and ‘Summary of findings' tables.
in: Higgins J.P.T. Green S. Cochrane Handbook for Systematic Reviews of Interventions Version 510. The Cochrane Collaboration,
West Sussex, UK2011: 2011(updated March 2011). Available at
Funding: Data collection for this study was supported by Ministry of Health, Saudi Arabia, and McMaster University, Hamilton, Canada. The sponsor had no role in the design of the study or interpretation of the results except for through the lead authors of this article.
Conflict of interest: Several authors are members of the GRADE working group and have helped developing the Evidence to Decision frameworks.
30482-6&id=gr1.jpg){kind=link}