Advertisement

GRADE equity guidelines 3: considering health equity in GRADE guideline development: rating the certainty of synthesized evidence

Open AccessPublished:April 04, 2017DOI:https://doi.org/10.1016/j.jclinepi.2017.01.015

      Abstract

      Objectives

      The aim of this paper is to describe a conceptual framework for how to consider health equity in the Grading Recommendations Assessment and Development Evidence (GRADE) guideline development process.

      Study Design and Setting

      Consensus-based guidance developed by the GRADE working group members and other methodologists.

      Results

      We developed consensus-based guidance to help address health equity when rating the certainty of synthesized evidence (i.e., quality of evidence). When health inequity is determined to be a concern by stakeholders, we propose five methods for explicitly assessing health equity: (1) include health equity as an outcome; (2) consider patient-important outcomes relevant to health equity; (3) assess differences in the relative effect size of the treatment; (4) assess differences in baseline risk and the differing impacts on absolute effects; and (5) assess indirectness of evidence to disadvantaged populations and/or settings.

      Conclusion

      The most important priority for research on health inequity and guidelines is to identify and document examples where health equity has been considered explicitly in guidelines. Although there is a weak scientific evidence base for assessing health equity, this should not discourage the explicit consideration of how guidelines and recommendations affect the most vulnerable members of society.

      Keywords

      What is new?

        Key findings

      • This paper provides consensus-based guidance for including health equity considerations in guideline development.

        What this study adds to what was known?

      • This paper adds an equity framework to the Grading Recommendations Assessment and Development Evidence (GRADE) guidance for rating the certainty of evidence in systematic reviews.

        What is the implication and what should change now?

      • Considering health equity in rating the certainty in synthesized evidence requires a priori elaboration of the disadvantaged populations and settings of interest, and methods to assess both relative and absolute effects for these populations.
      • GRADE judgements about directness require transparent reporting of how judgements were made.

      1. Background

      Health inequities are differences in health that are not only unnecessary and avoidable but are also considered unfair and unjust [
      • Whitehead M.
      The concepts and principles of equity and health.
      ]. As described in the introductory paper in this series, we use the acronym PROGRESS Plus (Place of residence, Race/ethnicity/culture/language, Occupation, Gender/sex, Religion, Education, Socioeconomic status, or Social capital + personal, relational and time-dependent characteristics) to identify individual and context-specific characteristics across which health inequities may occur [
      • O'Neill J.
      • Tabish H.
      • Welch V.
      • Petticrew M.
      • Pottie K.
      • Clarke M.
      • et al.
      Applying an equity lens to interventions: using PROGRESS ensures consideration of socially stratifying factors to illuminate inequities in health.
      ].
      Guideline panels need to decide early on whether they plan to develop equity-sensitive recommendations (as described in the introductory paper in this series). Using explicit prompts may be helpful in this process [
      • Welch V.
      • Akl E.A.
      • Guyatt G.
      • Pottie K.
      • Eslava-Schmalbache J.
      • Ansarif M.T.
      • et al.
      GRADE Equity: introduction: why health equity and GRADE guidelines?.
      ]. In principle, considering health equity is important for two main types of guidelines: (1) universal interventions where health inequity is a concern [
      National Institute for Health and Clinical Excellence
      Positively equal: a guide to addressing equality issues in developing NICE clinical guidelines.
      ,
      • Mitchell J.
      • Trangle M.
      • Degnan B.
      • Gabert T.
      • Haight B.
      • Kessler D.
      • et al.
      Depression, adult in primary care.
      ,
      • Eslava-Schmalbach J.
      • et al.
      Colombia guidelines on complications of pregnancy and childbirth.
      ,
      ]; and (2) targeted or dedicated interventions aimed at one or more disadvantaged populations that have experienced health inequities. An example of the latter is the Canadian immigrant health guidelines [
      • Pottie K.
      • Greenaway C.
      • Feightner J.
      • Welch V.
      • Swinkels H.
      • Rashid M.
      • et al.
      Evidence-based clinical guidelines for immigrants and refugees.
      ], developed to raise awareness of migrant health needs and improve access to effective preventive screening.
      This paper provides guidance to address health equity when rating the certainty in synthesized evidence using the Grading Recommendations Assessment and Development Evidence (GRADE) approach. This paper is the third paper in a four-part series on health equity and GRADE, with the introduction [Welch et al.], overall process [Akl et al.], and evidence to decision methods [Pottie et al.].

      2. Existing guidance

      As discussed in the earlier two papers in this series, several authors have assessed how guidelines consider health inequity concerns [Welch et al. this series, Akl et al. this series]. None of these papers focus on rating the certainty of synthesized evidence (i.e., quality of evidence) using the GRADE approach.

      3. GRADE certainty in synthesized evidence and health equity

      The GRADE approach of presenting the evidence by outcome and the associated certainty (i.e., quality of evidence) involves the production of summary tables. These tables include evidence profiles (with details on the rating of certainty for each outcome) and summary of finding (SoF) tables that are intended for the public, patients, purchasers, payers, practitioners, product makers (e.g., manufacturers, industry), and policy makers [
      • Concannon T.W.
      • Meissner P.
      • Grunbaum J.A.
      • McElwee N.
      • Guise J.M.
      • Santa J.
      • et al.
      A new taxonomy for stakeholder engagement in patient-centered outcomes research.
      ].
      Five methods can be used to assess health equity with the GRADE approach:
      • a)
        Include health equity as an outcome
      • b)
        Consider patient-important outcomes relevant to health equity
      • c)
        Assess differences in the magnitude of effect in relative terms between disadvantaged and more advantaged individuals or populations
      • d)
        Assess differences in baseline risk and hence the differing impacts on absolute effects for disadvantaged individuals or populations
      • e)
        Assess indirectness of evidence to disadvantaged populations and/or settings.

      3.1 Consider including health equity as an outcome for the SoF tables

      If health inequity is considered an important concern by relevant stakeholders, then health equity could be included as an outcome in the Population, Intervention, Comparison, Outcome questions, analytic framework, and SoF table. In doing so, guideline developers must recognize that health equity is primarily assessed with a subgroup analysis. The developers should also note that this may risk excluding other patient-important outcomes, if SoF tables are limited to only seven outcomes as recommended by GRADE. For example, the NICE guideline on maternal and child nutrition identified impact on health inequalities as one of its key priorities and framed its key question as: “What nutritional interventions are effective in improving the health of preconceptual, pregnant, and postpartum mothers and children (up to 5 years) and reducing nutrition-related health inequalities” [
      Maternal and child nutrition
      NICE public health guidance 11.
      ]. By including health equity as an outcome in the SoF table, it is easier for guideline panels to find the information (or lack thereof) about health equity and consider it in their deliberations.
      The direction and size of the effect on health equity is influenced by decisions such as the reference comparator group, use of relative or absolute measures, and whether the outcome is a desirable or undesirable event [
      • Harper S.
      • King N.B.
      • Young M.E.
      Impact of selective evidence presentation on judgments of health inequality trends: an experimental study.
      ,
      • Keppel K.
      • Pamuk E.
      • Lynch J.
      • Carter-Pokras O.
      • Kim I.
      • Mays V.
      • et al.
      Methodological issues in measuring health disparities.
      ]. For example, the choice of absolute or relative effects can change the conclusions about health inequalities. This is illustrated by gender disparity in stomach cancer mortality rates in the United States between 1930 and 2000 has decreased when looking at absolute differences (the rates for both men and women have declined). However, the relative risk for men compared to women has increased (increased disparity, male/female ratio) [
      • Harper S.
      • Lynch J.
      Methods for measuring cancer disparities: using data relevant to healthy people 2010 cancer-related objectives. Cancer surveillance monograph series, number 6.
      ].
      A lack of evidence about a critical health equity outcome should not be a reason to omit this from the SOF table. Indeed, this should be explicitly identified as an empty row, highlighting the need for further research to answer questions about health equity.

      3.1.1 Example 1

      The Community Guide Water fluoridation guideline [
      ] included “health disparities” as an outcome in the analytic framework and the SoF table because the Community Task Force placed a high value on reducing socioeconomic disparities in dental caries. Socioeconomic disparities were measured as the difference in absolute terms of a continuous outcome (caries). The evidence review found three studies that provided insufficient evidence about socioeconomic disparities to draw conclusions, highlighting a gap in the evidence base (Table 1).
      Table 1Effect of Community Water Fluoridation on socioeconomic health inequities in caries
      OutcomeMeasureEvidence
      Health equity as measured by socioeconomic disparities in caries% of caries reductionInconsistent results on socioeconomic disparities (three studies)
      dmft/DMFTNo data on socioeconomic disparities
      Abbreviation: DMFT/dmft, decayed, missing, or filled teeth.
      Upper case refers to permanent teeth; lower case to primary teeth.

      3.1.2 Example 2

      “Equity impact” was the primary outcome of a systematic review on interventions to reduce smoking in adults [
      • Brown T.
      • Platt S.
      • Amos A.
      Equity impact of population-level interventions and policies to reduce smoking in adults: a systematic review.
      ]. Equity impact was assessed as the difference in the magnitude of a dichotomous outcome in absolute terms, defined as a difference in absolute effect on prevalence in lower socioeconomic status compared to higher socioeconomic status. This review showed that while increases in price or taxes reduced health inequities in smoking, mass media campaigns were more likely to worsen health inequities. This type of review provides evidence that could be used to include impact on health equity as an outcome of interventions.

      3.2 Consider patient-important outcomes relevant to health equity

      As described in the previous paper in our series [Akl et al. in this series], the evidence synthesis process should consider the relative importance of different outcomes, determined with input from stakeholders representing disadvantaged groups. The evidence base for these outcomes should then be assessed. Examples of patient importance and health equity were provided in the previous paper in this series such as the importance of inconvenience of a subcutaneous chelation pump for people with sickle cell disease [described in Akl et al. in this series].

      3.3 Assess differences in the magnitude of effect in relative terms between disadvantaged and more advantaged individuals or populations

      Average effects obscure differences between subpopulations—that is, subgroup effects may exist. Examining whether effects differ across socioeconomic status or other variables relating to health inequity requires investigating heterogeneity in the treatment effect—for example, using statistical approaches such as meta-regression or subgroup analysis. However, such results may not be available in the literature. There is evidence that systematic reviews underreport subgroup analyses from primary studies [
      • Tugwell P.
      • Maxwell L.
      • Welch V.
      • Kristjansson E.
      • Petticrew M.
      • Wells G.
      • et al.
      Is health equity considered in systematic reviews of the Cochrane Musculoskeletal Group?.
      ,
      • Humphreys D.K.
      • Ogilvie D.
      Synthesising evidence for equity impacts of population-based physical activity interventions: a pilot study.
      ]. Furthermore, many primary studies fail to assess possible subgroup effects related to disadvantaged populations.
      Relative effects are usually similar across diverse populations and settings, and spurious subgroup effects are common [
      • Schmid C.H.
      • Lau J.
      • McIntosh M.W.
      • Cappelleri J.C.
      An empirical study of the effect of the control rate as a predictor of treatment efficacy in meta-analysis of clinical trials.
      ]. Thus, if analysis suggests an apparent subgroup effect, it is important to assess the credibility of the apparent effect [
      • Burke J.F.
      • Sussman J.B.
      • Kent D.M.
      • Hayward R.A.
      Three simple rules to ensure reasonably credible subgroup analyses.
      ]. Sun and colleagues [
      • Sun X.
      • Briel M.
      • Busse J.W.
      • You J.J.
      • Akl E.A.
      • Mejza F.
      • et al.
      Credibility of claims of subgroup effects in randomised controlled trials: systematic review.
      ,
      • Sun X.
      • Briel M.
      • Busse J.W.
      • You J.J.
      • Akl E.A.
      • Mejza F.
      • et al.
      The influence of study characteristics on reporting of subgroup analyses in randomised controlled trials: systematic review.
      ] describe several criteria to help do this such as determining a priori which subgroup analysis to conduct, finding a low P-value associated with a statistical test for interaction, and providing results from within-study comparisons. Sun et al. also showed that subgroup analyses reported in the literature rarely meet these criteria. Evidence synthesis that involves subgroup analyses should therefore consider the full set of credibility issues, using an appropriate checklist, and avoid making conclusions based on chance findings (Table 2).
      Table 2Checklist for assessing credibility of subgroup analyses
      • Sun X.
      • Briel M.
      • Walter S.D.
      • Guyatt G.H.
      Is a subgroup effect believable? Updating criteria to evaluate the credibility of subgroup analyses.
      • Design
        • Is the subgroup variable a characteristic measured at baseline or after randomization?
        • Is the effect suggested by comparisons within rather than between studies?
        • Was the hypothesis specified a priori?
        • Was the direction of the subgroup effect specified a priori?
        • Was the subgroup effect one of a small number of hypothesized effects tested?
      • Analysis
        • Does the interaction test suggest a low likelihood that chance explains the apparent subgroup effect?
        • Is the significant subgroup effect independent?
      • Context
        • Is the size of the subgroup effect large?
        • Is the interaction consistent across studies?
        • Is the interaction consistent across closely related outcomes within the study?
        • Is there indirect evidence that supports the hypothesized interaction (biological rationale)?
      If applying the criteria in Table 2 leads to a conclusion that the subgroup effect is credible, the guideline panel should provide different estimates of relative and absolute effect for the subgroups. The panel should then consider making different recommendations for patients in these subgroups or consider whether recommendations that apply to the overall population need to be adapted to enhance equity. When the credibility of subgroup effects is low, the guideline panel may suggest that further research is needed. Few subgroup analyses meet all of these criteria; however, when most criteria are met, decision making must consider the likely existence of subgroup effects.

      3.3.1 Example: hypertension and ethnicity

      The Eighth Joint National Committee guideline on management of hypertension recommends a calcium channel blocker or thiazide-type diuretic as initial therapy in the black hypertensive population (whereas an angiotensin-converting enzyme inhibitor, angiotensin receptor blocker, calcium channel blocker, or thiazide-type diuretic is recommended for others with hypertension) [
      • James P.A.
      • Oparil S.
      • Carter B.L.
      • Cushman W.C.
      • Dennison-Himmelfarb C.
      • Handler J.
      • et al.
      2014 evidence-based guideline for the management of high blood pressure in adults: report from the panel members appointed to the Eighth Joint National Committee (JNC 8).
      ]. This recommendation was based on a prespecified subgroup analysis of the ALLHAT trial (n = 18,102 participants, 35% black [
      • Leenen F.H.
      • Nwachuku C.E.
      • Black H.R.
      • Cushman W.C.
      • Davis B.R.
      • Simpson L.M.
      • et al.
      Clinical events in high-risk hypertensive patients randomly assigned to calcium channel blocker versus angiotensin-converting enzyme inhibitor in the antihypertensive and lipid-lowering treatment to prevent heart attack trial.
      ]) that showed stroke was 51% (95% CI: 1.22, 1.86) greater for blacks treated with an ACE inhibitor first compared to those treated with a calcium channel blocker. The guideline panel rated this subgroup effect as moderate quality evidence. Had the panel not identified this subgroup effect, use of an ACE inhibitor as a first-line agent would have increased health disparities between black and white ethnic groups.

      3.4 Assess differences in baseline risk and the differing impacts on absolute effects for disadvantaged individuals or populations

      A higher baseline risk of adverse events in any population may lead to greater absolute harm from an intervention and conversely a higher baseline prevalence of the outcome of interest may lead to greater absolute benefit [
      • Manuel D.G.
      • Ho T.H.
      • Harper S.
      • Anderson G.M.
      • Lynch J.
      • Rosella L.C.
      Modelling preventive effectiveness to estimate the equity tipping point: at what coverage can individual preventive interventions reduce socioeconomic disparities in diabetes risk?.
      ]. The SoF table should present the baseline risks and risk differences for each relevant population and provide supporting evidence. Because disadvantaged populations have a disproportionate burden of almost all health conditions, it is particularly important to consider the baseline risk for these populations. Baseline risk of adverse event rates or for the outcomes of interest for specific populations are best assessed using the most robust observational data on the actual population rather than from randomized trials. GRADE guidance regarding assessing certainty of estimates of risk from broad populations is available [
      • Spencer F.A.
      • Iorio A.
      • You J.
      • Murad M.H.
      • Schunemann H.J.
      • Vandvik P.O.
      • et al.
      Uncertainties in baseline risk estimates and confidence in treatment effects.
      ,
      • Iorio A.
      • Spencer F.A.
      • Falavigna M.
      • Alba C.
      • Lang E.
      • Burnand B.
      • et al.
      Use of GRADE for assessment of evidence about prognosis: rating confidence in estimates of event rates in broad categories of patients.
      ].

      3.4.1 Example 1: WHO guidelines on vitamin A supplementation in children 6–59 months

      In 2011, WHO recommended vitamin A supplementation for children aged 6 months to 5 years in countries where vitamin A deficiency is a public health problem (strong recommendation) [
      World Health Organization
      Guideline: vitamin A supplementation in infants and children 6-59 months of age.
      ]. This was based on findings of a Cochrane review with a relative risk for all-cause mortality of 0.76 (95% CI: 0.69, 0.83). The baseline risk of all-cause mortality was estimated at 0/1,000 in low-risk populations and 90/1,000 in high-risk populations (with vitamin A deficiency), based on control group event rates in the trials. Thus, the absolute effects in terms of numbers of deaths prevented with vitamin A compared to the control group were 0/1,000 for low-risk and 22/1,000 for high-risk populations.

      3.4.2 Example 2: national guide to a preventive health assessment for Aboriginal and Torres Strait Islander people

      In Australia, a guideline panel sought to determine the optimal age at which to begin a series of preventive interventions in the Australian Aboriginal and Torres Strait Islander population. The panel recommended preventive interventions at an earlier age than the general population on the basis of higher prevalence of preventable diseases in Aboriginal and Torres Strait Islander populations. For example, type II diabetes is 3–4 times more common than in the general Australian population at all ages, leading to a recommendation for screening starting from age 18, instead of age 40 years for the general population [
      NACCHO/RACGP
      National guide to a preventive health assessment for Aboriginal and Torres Strait Islander people.
      ].

      3.5 Assess indirectness of evidence to disadvantaged populations

      GRADE quality (or certainty) “reflects our confidence that the estimates of the effect are correct. In the context of recommendations, quality reflects our confidence that the effect estimates are adequate to support a particular recommendation. ‘Quality’ as used in GRADE means more than risk of bias and so may also be compromised by imprecision, inconsistency, indirectness of study results, and publication bias” [
      • Balshem H.
      • Helfand M.
      • Schunemann H.J.
      • Oxman A.D.
      • Kunz R.
      • Brozek J.
      • et al.
      GRADE guidelines: 3. Rating the quality of evidence.
      ]. Qualitative evidence may also be important when considering health equity. Certainty for qualitative evidence synthesis can be rated using the CerQUAL tool [
      • Lewin S.
      • Glenton C.
      • Munthe-Kaas H.
      • Carlsen B.
      • Colvin C.J.
      • Gulmezoglu M.
      • et al.
      Using qualitative evidence in decision making for health and social interventions: an approach to assess confidence in findings from qualitative evidence syntheses (GRADE-CERQual).
      ] in which the domain “relevance” is most closely aligned with directness.
      Indirectness refers to the comparability between the population, the intervention, or the outcomes measured in research studies and those under consideration in a guideline or systematic review [
      • Guyatt G.H.
      • Oxman A.D.
      • Kunz R.
      • Woodcock J.
      • Brozek J.
      • Helfand M.
      • et al.
      GRADE guidelines: 8. Rating the quality of evidence–indirectness.
      ]. The GRADE approach evaluates the lack of directness as “indirectness.” Direct evidence may be lacking because some populations may not represent a large proportion of trial populations (e.g., migrants and refugees), and data are unlikely to be disaggregated for specific subgroups. Direct evidence may also be lacking because some populations are explicitly excluded from trials, such as pregnant women and people with multiple morbidities [
      • Bartlett C.
      • Doyal L.
      • Ebrahim S.
      • Davey P.
      • Bachmann M.
      • Egger M.
      • et al.
      The causes and effects of socio-demographic exclusions from clinical trials.
      ,
      • Braun J.
      • Brandt J.
      • Listing J.
      • Zink A.
      • Alten R.
      • Golder W.
      • et al.
      Treatment of active ankylosing spondylitis with infliximab: a randomised controlled multicentre trial.
      ,
      • O'Dell J.R.
      • Mikuls T.R.
      • Taylor T.H.
      • Ahluwalia V.
      • Brophy M.
      • Warren S.R.
      • et al.
      Therapies for active rheumatoid arthritis after methotrexate failure.
      ,
      • Charytan D.
      • Kuntz R.E.
      The exclusion of patients with chronic kidney disease from clinical trials in coronary artery disease.
      ]. Because multiple morbidities are more common in socioeconomically disadvantaged people [
      • Barnett K.
      • Mercer S.W.
      • Norbury M.
      • Watt G.
      • Wyke S.
      • Guthrie B.
      Epidemiology of multimorbidity and implications for health care, research, and medical education: a cross-sectional study.
      ], this may result in disproportionate exclusion of disadvantaged populations from trials. When direct evidence for the relevant disadvantaged population is not available, guideline developers will have to evaluate the indirectness of evidence obtained from other populations [
      • Welch V.
      • Brand K.
      • Kristjansson E.
      • Smylie J.
      • Wells G.
      • Tugwell P.
      Systematic reviews need to consider applicability to disadvantaged populations: inter-rater agreement for a health equity plausibility algorithm.
      ].
      As a rule, certainty of the evidence should not be rated down for indirectness for population differences unless there are compelling reasons to anticipate differences in effect due to biology/physiology, sociocultural influences, or setting-specific resource issues that impact the effectiveness or harms of the intervention. In other words, one anticipates a different subgroup effect in either relative or absolute impact of treatment, though evidence is not available to make a formal assessment. (If it were, it should be formally assessed, as in Sections 3.3 and 3.4) Guideline panels need to consider that rating down for indirectness could in itself increase inequities if this leads to less use of an effective intervention by disadvantaged groups. In other words, lower certainty in effect estimates may lead to a weak recommendation and therefore under-use of a beneficial treatment. Rating down for indirectness should therefore be done cautiously because effective interventions are needed even more in some populations that are often excluded from trials, such as those with multiple morbidities.

      3.5.1 Example 1: Canadian migrant guidelines not rated down for indirectness

      The quality of the evidence was not rated down for indirectness in the Canadian migrant guideline addressing screening for latent TB; the panel considered the evidence not to be indirect for migrants. Although no migrants were included in studies of intervention effectiveness, the developers did not expect different relative effects [
      • Greenaway C.
      • Sandoe A.
      • Vissandjee B.
      • Kitai I.
      • Gruner D.
      • Wobeser W.
      • et al.
      Tuberculosis: evidence review for newly arriving immigrants and refugees.
      ].

      3.5.2 Example 2: CDC guidelines for brief alcohol counseling for people with HCV infection rated down for indirectness

      The Centers for Disease Control and Prevention recommended brief alcohol screening and counseling for all person with HCV infection, based on a systematic review of 22 randomized trials which found a reduction of alcohol consumption of 38.42% (95% CI: 30.91, 65.44) more than the control groups after 1 year. This evidence was rated down for indirectness by the guideline panel because none of the trials included persons with hepatitis C virus (HCV) infection [
      • Smith B.D.
      • Morgan R.L.
      • Beckett G.A.
      • Falck-Ytter Y.
      • Holtzman D.
      • Teo C.G.
      • et al.
      Recommendations for the identification of chronic hepatitis C virus infection among persons born during 1945–1965.
      ].

      4. Methodologic challenges

      In developing this guidance, we identified a number of methodologic challenges. First, assessing effects on health equity is not a linear process. There may be a need to revisit the focus of the guideline during the evidence review process, including the consideration of important disadvantaged groups. NICE does this explicitly by revisiting their key questions regarding health equity throughout the process.
      Second, there are often limitations in the underlying evidence base including poor reporting of sociodemographic characteristics [
      • Furler J.
      • Magin P.
      • Pirotta M.
      • van Driel M.
      Participant demographics reported in “Table 1” of randomised controlled trials: a case of “inverse evidence”?.
      ,
      • Welch V.
      • Tugwell P.
      • Petticrew M.
      • de Montigny J.
      • Ueffing E.
      • Kristjansson B.
      • et al.
      How effects on health equity are assessed in systematic reviews of interventions.
      ], under-reporting of subgroup analyses that are not statistically significant [
      • Sun X.
      • Briel M.
      • Busse J.W.
      • You J.J.
      • Akl E.A.
      • Mejza F.
      • et al.
      The influence of study characteristics on reporting of subgroup analyses in randomised controlled trials: systematic review.
      ,
      • Welch V.
      • Tugwell P.
      • Petticrew M.
      • de Montigny J.
      • Ueffing E.
      • Kristjansson B.
      • et al.
      How effects on health equity are assessed in systematic reviews of interventions.
      ], and use of multivariable models that may be overadjusted for effect mediators, and/or include unnecessary collinear variables [
      • Morton R.L.
      • Schlackow I.
      • Mihaylova B.
      • Staplin N.D.
      • Gray A.
      • Cass A.
      The impact of social disadvantage in moderate-to-severe chronic kidney disease: an equity-focused systematic review.
      ]. Lack of evidence on whether the effects are consistent or different for disadvantaged populations makes it difficult to judge indirectness and rate certainty of evidence. When the evidence base is insufficient to assess effects on health equity, guideline panels need to make these limitations explicit and transparently report how they made judgments.
      Third, epidemiologic evidence addressing baseline risk for specific disadvantaged groups may be difficult to obtain for the population or geographic region for which the recommendations are being developed. Health systems at local, regional, and national levels do not have consistent or reliable methods for reporting health status across all sociodemographic indicators of interest. Guideline panels should transparently report how they determined baseline risk estimates.
      Fourth, assessing directness of evidence depends on the clinical and methodological expertise and judgment of SoF developers. The GRADE Guideline Development Tool includes an explicit checklist when producing SoFs to ask whether the evidence is direct across population, intervention, comparison, and outcome and document the decision for rating down, if performed.

      5. Research agenda

      The most important research priority in the field of health equity and guidelines is to systematically identify further examples of how guideline panels have assessed health equity considerations and incorporated these assessments into recommendations using transparent methods. For example, all WHO guidelines make their evidence to recommendation tables and SoFs publicly available for research such as this. These assessments could provide examples of whether, and how, the five issues (a–e) above have been considered for different situations, such as assessing the credibility of subgroup analyses and judging indirectness for disadvantaged populations.
      In conclusion, the GRADE process provides a structured approach to assess effects on health equity. Health equity considerations warrant increased use of these methods in systematic reviews and guidelines. The findings of assessing health equity using these five steps in guideline development provides a basis for judging “impact on equity” which is part of the DECIDE framework, and details about this process are covered in the fourth paper of this series [Pottie et al.].

      Acknowledgments

      The authors gratefully acknowledge the help from Manosila Yoganathan and Shalini Suresh in preparing this manuscript for submission.

      References

        • Whitehead M.
        The concepts and principles of equity and health.
        Int J Health Serv. 1992; 22: 429-445
        • O'Neill J.
        • Tabish H.
        • Welch V.
        • Petticrew M.
        • Pottie K.
        • Clarke M.
        • et al.
        Applying an equity lens to interventions: using PROGRESS ensures consideration of socially stratifying factors to illuminate inequities in health.
        J Clin Epidemiol. 2014; 67: 56-64
        • Welch V.
        • Akl E.A.
        • Guyatt G.
        • Pottie K.
        • Eslava-Schmalbache J.
        • Ansarif M.T.
        • et al.
        GRADE Equity: introduction: why health equity and GRADE guidelines?.
        J Clin Epidemiol. 2016;
        • National Institute for Health and Clinical Excellence
        Positively equal: a guide to addressing equality issues in developing NICE clinical guidelines.
        2nd ed. National Institute for Health and Clinical Excellence, London, UK2012 (Available at http://www.nice.org.uk. Accessed June 1, 2017)
        • Mitchell J.
        • Trangle M.
        • Degnan B.
        • Gabert T.
        • Haight B.
        • Kessler D.
        • et al.
        Depression, adult in primary care.
        Institute for Clinical Systems Improvement (ICSI), 2013 (Available at https://www.icsi.org/guidelines__more/catalog_guidelines_and_more/catalog_guidelines/catalog_behavioral_health_guidelines/depression/. Accessed June 1, 2017)
        • Eslava-Schmalbach J.
        • et al.
        Colombia guidelines on complications of pregnancy and childbirth.
        2013
      1. Suspected cancer in primary care: guidelines for investigation, referral and reducing ethnic disparities. New Zealand Guidelines Grup, Wellington, NZ2009 (Available at http://www.health.govt.nz/system/files/documents/publications/suspected-cancer-guideline-sep09.pdf. Accessed June 1, 2017)
        • Pottie K.
        • Greenaway C.
        • Feightner J.
        • Welch V.
        • Swinkels H.
        • Rashid M.
        • et al.
        Evidence-based clinical guidelines for immigrants and refugees.
        CMAJ. 2011; 183: E824-E925
        • Concannon T.W.
        • Meissner P.
        • Grunbaum J.A.
        • McElwee N.
        • Guise J.M.
        • Santa J.
        • et al.
        A new taxonomy for stakeholder engagement in patient-centered outcomes research.
        J Gen Intern Med. 2012; 27: 985-991
        • Maternal and child nutrition
        NICE public health guidance 11.
        NICE, London, UK2014 (Available at https://www.nice.org.uk/guidance/ph11. Accessed June 1, 2017)
        • Harper S.
        • King N.B.
        • Young M.E.
        Impact of selective evidence presentation on judgments of health inequality trends: an experimental study.
        PLoS One. 2013; 8: e63362
        • Keppel K.
        • Pamuk E.
        • Lynch J.
        • Carter-Pokras O.
        • Kim I.
        • Mays V.
        • et al.
        Methodological issues in measuring health disparities.
        Vital Health Stat 2. 2005; : 1-16
        • Harper S.
        • Lynch J.
        Methods for measuring cancer disparities: using data relevant to healthy people 2010 cancer-related objectives. Cancer surveillance monograph series, number 6.
        National Cancer Institute, Bethesda, MD2005 (Available at http://seer.cancer.gov/archive/publications/disparities/measuring_disparities.pdf. Accessed June 1, 2017)
      2. Preventing dental caries: community water fluoridation. The community guide community preventive services task force. 2013 (Atlanta)
        • Brown T.
        • Platt S.
        • Amos A.
        Equity impact of population-level interventions and policies to reduce smoking in adults: a systematic review.
        Drug Alcohol Depend. 2014; 138: 7-16
        • Tugwell P.
        • Maxwell L.
        • Welch V.
        • Kristjansson E.
        • Petticrew M.
        • Wells G.
        • et al.
        Is health equity considered in systematic reviews of the Cochrane Musculoskeletal Group?.
        Arthritis Rheum. 2008; 59: 1603-1610
        • Humphreys D.K.
        • Ogilvie D.
        Synthesising evidence for equity impacts of population-based physical activity interventions: a pilot study.
        Int J Behav Nutr Phys Act. 2013; 10: 76
        • Schmid C.H.
        • Lau J.
        • McIntosh M.W.
        • Cappelleri J.C.
        An empirical study of the effect of the control rate as a predictor of treatment efficacy in meta-analysis of clinical trials.
        Stat Med. 1998; 17: 1923-1942
        • Burke J.F.
        • Sussman J.B.
        • Kent D.M.
        • Hayward R.A.
        Three simple rules to ensure reasonably credible subgroup analyses.
        BMJ. 2015; 351: h5651
        • Sun X.
        • Briel M.
        • Busse J.W.
        • You J.J.
        • Akl E.A.
        • Mejza F.
        • et al.
        Credibility of claims of subgroup effects in randomised controlled trials: systematic review.
        BMJ. 2012; 344: e1553
        • Sun X.
        • Briel M.
        • Busse J.W.
        • You J.J.
        • Akl E.A.
        • Mejza F.
        • et al.
        The influence of study characteristics on reporting of subgroup analyses in randomised controlled trials: systematic review.
        BMJ. 2011; 342: d1569
        • Sun X.
        • Briel M.
        • Walter S.D.
        • Guyatt G.H.
        Is a subgroup effect believable? Updating criteria to evaluate the credibility of subgroup analyses.
        BMJ. 2010; 340: c117
        • James P.A.
        • Oparil S.
        • Carter B.L.
        • Cushman W.C.
        • Dennison-Himmelfarb C.
        • Handler J.
        • et al.
        2014 evidence-based guideline for the management of high blood pressure in adults: report from the panel members appointed to the Eighth Joint National Committee (JNC 8).
        JAMA. 2014; 311: 507-520
        • Leenen F.H.
        • Nwachuku C.E.
        • Black H.R.
        • Cushman W.C.
        • Davis B.R.
        • Simpson L.M.
        • et al.
        Clinical events in high-risk hypertensive patients randomly assigned to calcium channel blocker versus angiotensin-converting enzyme inhibitor in the antihypertensive and lipid-lowering treatment to prevent heart attack trial.
        Hypertension. 2006; 48: 374-384
        • Manuel D.G.
        • Ho T.H.
        • Harper S.
        • Anderson G.M.
        • Lynch J.
        • Rosella L.C.
        Modelling preventive effectiveness to estimate the equity tipping point: at what coverage can individual preventive interventions reduce socioeconomic disparities in diabetes risk?.
        Chronic Dis Inj Can. 2014; 34: 94-102
        • Spencer F.A.
        • Iorio A.
        • You J.
        • Murad M.H.
        • Schunemann H.J.
        • Vandvik P.O.
        • et al.
        Uncertainties in baseline risk estimates and confidence in treatment effects.
        BMJ. 2012; 345: e7401
        • Iorio A.
        • Spencer F.A.
        • Falavigna M.
        • Alba C.
        • Lang E.
        • Burnand B.
        • et al.
        Use of GRADE for assessment of evidence about prognosis: rating confidence in estimates of event rates in broad categories of patients.
        BMJ. 2015; 350: h870
        • World Health Organization
        Guideline: vitamin A supplementation in infants and children 6-59 months of age.
        World Health Organization, Geneva, Switzerland2011 (Available at) (Accessed June 1, 2017)
        • NACCHO/RACGP
        National guide to a preventive health assessment for Aboriginal and Torres Strait Islander people.
        2nd ed. National Aboriginal Community Controlled Health Organisation (NACCHO) and The Royal Australian College of General Practitioners (RACGP), South Melbourne2012 (Available at http://www.racgp.org.au/download/documents/AHU/2ndednationalguide.pdf. Accessed June 1, 2017)
        • Balshem H.
        • Helfand M.
        • Schunemann H.J.
        • Oxman A.D.
        • Kunz R.
        • Brozek J.
        • et al.
        GRADE guidelines: 3. Rating the quality of evidence.
        J Clin Epidemiol. 2011; 64: 401-406
        • Lewin S.
        • Glenton C.
        • Munthe-Kaas H.
        • Carlsen B.
        • Colvin C.J.
        • Gulmezoglu M.
        • et al.
        Using qualitative evidence in decision making for health and social interventions: an approach to assess confidence in findings from qualitative evidence syntheses (GRADE-CERQual).
        PLoS Med. 2015; 12: e1001895
        • Guyatt G.H.
        • Oxman A.D.
        • Kunz R.
        • Woodcock J.
        • Brozek J.
        • Helfand M.
        • et al.
        GRADE guidelines: 8. Rating the quality of evidence–indirectness.
        J Clin Epidemiol. 2011; 64: 1303-1310
        • Bartlett C.
        • Doyal L.
        • Ebrahim S.
        • Davey P.
        • Bachmann M.
        • Egger M.
        • et al.
        The causes and effects of socio-demographic exclusions from clinical trials.
        Health Technol Assess. 2005; 9: 1-152
        • Braun J.
        • Brandt J.
        • Listing J.
        • Zink A.
        • Alten R.
        • Golder W.
        • et al.
        Treatment of active ankylosing spondylitis with infliximab: a randomised controlled multicentre trial.
        Lancet. 2002; 359: 1187-1193
        • O'Dell J.R.
        • Mikuls T.R.
        • Taylor T.H.
        • Ahluwalia V.
        • Brophy M.
        • Warren S.R.
        • et al.
        Therapies for active rheumatoid arthritis after methotrexate failure.
        N Engl J Med. 2013; 369: 307-318
        • Charytan D.
        • Kuntz R.E.
        The exclusion of patients with chronic kidney disease from clinical trials in coronary artery disease.
        Kidney Int. 2006; 70: 2021-2030
        • Barnett K.
        • Mercer S.W.
        • Norbury M.
        • Watt G.
        • Wyke S.
        • Guthrie B.
        Epidemiology of multimorbidity and implications for health care, research, and medical education: a cross-sectional study.
        Lancet. 2012; 380: 37-43
        • Welch V.
        • Brand K.
        • Kristjansson E.
        • Smylie J.
        • Wells G.
        • Tugwell P.
        Systematic reviews need to consider applicability to disadvantaged populations: inter-rater agreement for a health equity plausibility algorithm.
        BMC Med Res Methodol. 2012; 12: 187
        • Greenaway C.
        • Sandoe A.
        • Vissandjee B.
        • Kitai I.
        • Gruner D.
        • Wobeser W.
        • et al.
        Tuberculosis: evidence review for newly arriving immigrants and refugees.
        CMAJ. 2011; 183: E939-E951
        • Smith B.D.
        • Morgan R.L.
        • Beckett G.A.
        • Falck-Ytter Y.
        • Holtzman D.
        • Teo C.G.
        • et al.
        Recommendations for the identification of chronic hepatitis C virus infection among persons born during 1945–1965.
        MMWR Recomm Rep. 2012; 61: 1-32
        • Furler J.
        • Magin P.
        • Pirotta M.
        • van Driel M.
        Participant demographics reported in “Table 1” of randomised controlled trials: a case of “inverse evidence”?.
        Int J Equity Health. 2012; 11: 14
        • Welch V.
        • Tugwell P.
        • Petticrew M.
        • de Montigny J.
        • Ueffing E.
        • Kristjansson B.
        • et al.
        How effects on health equity are assessed in systematic reviews of interventions.
        Cochrane Database Syst Rev. 2010; : MR000028
        • Morton R.L.
        • Schlackow I.
        • Mihaylova B.
        • Staplin N.D.
        • Gray A.
        • Cass A.
        The impact of social disadvantage in moderate-to-severe chronic kidney disease: an equity-focused systematic review.
        Nephrol Dial Transplant. 2016; 31: 46-56