Advertisement

Quality assessment of prevalence studies: a systematic review

  • Celina Borges Migliavaca
    Correspondence
    Corresponding author. Tel.: +55 51 997107769; fax: +55 51 35378347.
    Affiliations
    Programa de Pós-Graduação em Epidemiologia, Universidade Federal do Rio Grande do Sul, Rua Ramiro Barcelos, 2400, CEP 90035-003, Santa Cecília, Porto Alegre, Rio Grande do Sul, Brazil

    Hospital Moinhos de Vento, Porto Alegre, Rua Ramiro Barcelos, 910, CEP 90035-001, Floresta, Porto Alegre, Rio Grande do Sul, Brazil
    Search for articles by this author
  • Cinara Stein
    Affiliations
    Hospital Moinhos de Vento, Porto Alegre, Rua Ramiro Barcelos, 910, CEP 90035-001, Floresta, Porto Alegre, Rio Grande do Sul, Brazil
    Search for articles by this author
  • Verônica Colpani
    Affiliations
    Hospital Moinhos de Vento, Porto Alegre, Rua Ramiro Barcelos, 910, CEP 90035-001, Floresta, Porto Alegre, Rio Grande do Sul, Brazil
    Search for articles by this author
  • Zachary Munn
    Affiliations
    Joanna Briggs Institute, Faculty of Health Sciences, University of Adelaide, Adelaide, Australia
    Search for articles by this author
  • Maicon Falavigna
    Affiliations
    Programa de Pós-Graduação em Epidemiologia, Universidade Federal do Rio Grande do Sul, Rua Ramiro Barcelos, 2400, CEP 90035-003, Santa Cecília, Porto Alegre, Rio Grande do Sul, Brazil

    Hospital Moinhos de Vento, Porto Alegre, Rua Ramiro Barcelos, 910, CEP 90035-001, Floresta, Porto Alegre, Rio Grande do Sul, Brazil
    Search for articles by this author
  • Prevalence Estimates Reviews – Systematic Review Methodology Group (PERSyst)
Open AccessPublished:July 14, 2020DOI:https://doi.org/10.1016/j.jclinepi.2020.06.039

      Abstract

      Objectives

      The objective of the study is to identify items and domains applicable for the quality assessment of prevalence studies.

      Study Design and Setting

      We searched databases and the gray literature to identify tools or guides about the quality assessment of prevalence studies. After study selection, we abstracted questions applicable for prevalence studies and classified into at least one of the following domains: ‘population and setting’, ‘condition measurement’, ‘statistics’, and ‘other’. PROSPERO registration:CRD42018088437.

      Results

      We included 30 tools: eight (26.7%) specifically designed to appraise prevalence studies and 22 (73.3%) adaptable for this purpose. We identified 12 unique items in the domain “population and setting”, 16 in the domain “condition measurement”, and 14 in the domain “statistics”. Of those, 25 (59.5%) were identified in the eight specific tools. Regarding the domain “other”, we identified 77 unique items, mainly related to manuscript writing and reporting (n = 48, 62.3%); of those, 24 (31.2%) were identified in the eight specific tools and 53 (68.8%) in the additional 22 nonspecific tools.

      Conclusion

      We provide a comprehensive set of items classified by domains that can guide the appraisal of prevalence studies, conduction of primary prevalence studies, and update or development of tools to evaluate prevalence studies.

      Keywords

      what is new section

        Key findings

      • We systematically reviewed tools used to assess risk of bias of prevalence studies.
      • We identified 30 tools; eight of them were specifically designed for prevalence studies

        What this adds to what was known?

      • There was a great variability among items assessed in each tool.
      • Not all tools assessed all domains, and there was overlap among items in some tools.

        What is the implication and what should change now?

      • We provide a comprehensive set of items useful to appraise prevalence studies.

      1. Background

      Prevalence is an epidemiological measurement that represents the proportion of the population affected by certain condition [
      • Fletcher R.
      • Fletcher S.
      • Fletcher G.S.
      Clinical epidemiology: the essentials.
      ]. Because they reflect the importance of different diseases for the society, prevalence estimates are of great importance for health-related decision-making. For instance, these estimates are used to assess the burden of different conditions, helping in the definition of priorities for interventions, guideline development, and research. They are also useful to evaluate the impact of health interventions because they show changes and trends over time in conditions of interest. For health technology assessments, prevalence data are also used in the estimation of costs, being an essential parameter in economic models [
      • Harder T.
      Some notes on critical appraisal of prevalence studies: comment on: "The development of a critical appraisal tool for use in systematic reviews addressing questions of prevalence.
      ,
      • Wagner M.B.
      Medindo a corrêencia da doença: prevalência ou incidência?.
      ,
      • Oxman A.D.
      • Schunemann H.J.
      • Fretheim A.
      Improving the use of research evidence in guideline development: 2. Priority setting.
      ,
      • Rotily M.
      • Roze S.
      What is the impact of disease prevalence upon health technology assessment?.
      ]. The number of systematic reviews of prevalence indexed in Medline has increased more than ten-fold in the last decade [
      • Borges Migliavaca C.
      • Stein C.
      • Colpani V.
      • et al.
      How are systematic reviews of prevalence conducted? A methodological study.
      ].
      Despite the importance of prevalence studies, the risk of bias assessment of this type of study is heterogeneous, usually inappropriate and often neglected. A systematic review conducted in 2010 identified five tools specifically developed to appraise prevalence studies, and the authors of that review concluded that included tools presented several limitations specially regarding applicability and lack of consensus about which domains should be assessed [
      • Shamliyan T.
      • Kane R.L.
      • Dickinson S.
      A systematic review of tools used to assess the quality of observational studies that examine incidence or prevalence and risk factors for diseases.
      ]. In comparison, there are standard, recommended, and widely used tools for other study designs, such as RoB 2.0 for randomized clinical trials, ROBINS-I for observational studies, and QUADAS-2 for diagnostic studies [
      • Sterne J.A.
      • Hernan M.A.
      • Reeves B.C.
      • Savovic J.
      • Berkman N.D.
      • Viswanathan M.
      • et al.
      ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions.
      ,
      • Whiting P.F.
      • Rutjes A.W.
      • Westwood M.E.
      • Mallett S.
      • Deeks J.J.
      • Reitsma J.B.
      • et al.
      QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies.
      ,
      • Sterne J.A.C.
      • Page M.J.
      • Elbers R.G.
      • Blencowe N.S.
      • Boutron I.
      • Cates C.J.
      • et al.
      RoB 2: a revised tool for assessing risk of bias in randomised trials.
      ].
      In light of the above, the objective of this study is to systematically review, evaluate, and compare available tools designed to assess the risk of bias of prevalence studies in order to identify the domains and items used to evaluate this type of study, providing information that could be used in the development and update of tools, critical appraisal of this type of study, and conduction of primary studies of prevalence.

      2. Methods

      2.1 Study design, protocol, and registration

      The present study is a systematic review. The study protocol was registered on PROSPERO, under the registration number CRD42018088437.

      2.2 Search strategy and data sources

      We searched Medline (via PubMed), Embase, and Web of Science up to August 2019 using terms such as “prevalence”, “cross-sectional studies”, and “critical appraisal”. The complete search strategy is presented in Additional file 1. The search was not limited by date or language of publication.
      To identify studies not indexed by these databases, we also screened the first 200 results on Google Scholar. We also manually searched the reference list of relevant studies and searched for instruments on websites of institutions related to the topic. Moreover, we conducted a systematic search for systematic reviews of prevalence of clinical conditions published between February 2017 and February 2018 and indexed in Medline to identify further instruments used to assess the quality of individual prevalence studies [
      • Borges Migliavaca C.
      • Stein C.
      • Colpani V.
      • et al.
      How are systematic reviews of prevalence conducted? A methodological study.
      ].
      For each instrument found, we conducted an internet search for complementary material, including handbooks or manuals for the instrument in question.

      2.3 Study selection

      We included methodological studies, manuals, or handbooks with general guidance or specific tools applicable for the critical appraise of prevalence studies. First, we reviewed the title and abstracts of all records identified in our search to select all potentially relevant studies. Then, we assessed the full text of selected studies and included studies meeting the eligibility criteria. Study selection was conducted by two reviewers independently (C.B.M. and C.S.). Disagreements were solved by consensus or arbitrated by a third reviewer (V.C. or M.F.).
      A tool was eligible if (1) it was developed to critically appraise prevalence studies or (2) the authors stated it could be applied to appraise prevalence studies or (3) it was used by systematic review authors to appraise the quality of individual prevalence studies.

      2.4 Data extraction

      We extracted relevant information for each tool using predesigned and piloted tables. Data extracted included: process of development, applicability, structure, and content of the tool. Data extraction was conducted by two independent reviewers (C.B.M. and C.S.). Disagreements were solved by consensus or arbitrated by a third reviewer (V.C. or M.F.).

      2.5 Data analysis

      We classified each question or statement of the instruments into items, which represented the objective of assessment. The items were not prespecified. Each item is unique and based on the fact that it addresses a different aspect of quality of the study under appraisal. A new item was created whenever the question/statement from the included instrument would represent a different aspect of risk of bias, trying to be as sensitive as possible. Questions or statements from different instruments (sometimes even from the same instrument) that assessed the same risk of bias aspect, but with different wording, were merged under the same item. The judgments regarding the classification of questions and statements into items and domains are available in Additional file 5. Afterward, we classified the items into three key domains: “population and setting”, “condition measurement”, and “statistics”. The domains were defined a priori based on the main components of a prevalence research question (population and condition) and considering the importance of appropriate statistical data analysis. If a question was applicable to appraise prevalence studies but covered a different domain (such as reporting or study methods), it was included under the classification “other”.
      As described, we included not only tools specifically designed to assess prevalence studies but also tools that could be adapted for this purpose. Thus, not all questions from nonspecific tools were applicable for prevalence studies (such as questions assessing the comparability among groups, or the description of intervention), and we only categorized the applicable ones. If the instrument provided guidance about which questions should be used to assess prevalence studies, we followed these instructions. If not, before classyifing the questions into items and domains, we evaluated if they were applicable for prevalence studies or not. If classified as applicable, the question was categorized into items and domains as previously described. Questions classified as not applicable were not further evaluated (Fig. 1).
      Figure thumbnail gr1
      Fig. 1Flowchart of classification of questions/statements. ∗In some instruments, the guidance about which questions/statements were applicable for prevalence studies was based on study aspects such as intervention or the comparison group.
      The process of selection and classification of questions/statements into items and domains was conducted by two reviewers independently (C.B.M. and C.S.). Discrepancies were solved by consensus or arbitrated by a third reviewer (V.C. or M.F.).

      3. Results

      3.1 Study selection

      Our search resulted in 1,690 unique references. After selection of titles and abstracts, we assessed 105 full texts for eligibility. Finally, we included in the review 30 tools [
      • Al-Jader L.N.
      • Newcombe R.G.
      • Hayes S.
      • Murray A.
      • Layzell J.
      • Harper P.S.
      Developing a quality scoring system for epidemiological surveys of genetic disorders.
      ,
      • Boyle M.H.
      Guidelines for evaluating prevalence studies.
      ,
      • Giannakopoulos N.N.
      • Rammelsberg P.
      • Eberhard L.
      • Schmitter M.
      A new instrument for assessing the quality of studies on prevalence.
      ,
      • Hoy D.
      • Brooks P.
      • Woolf A.
      • Blyth F.
      • March L.
      • Bain C.
      • et al.
      Assessing risk of bias in prevalence studies: modification of an existing tool and evidence of interrater agreement.
      ,
      • Loney P.L.
      • Chambers L.W.
      • Bennett K.J.
      • Roberts J.G.
      • Stratford P.W.
      Critical appraisal of the health research literature: prevalence or incidence of a health problem.
      ,
      • Shamliyan T.A.
      • Kane R.L.
      • Ansari M.T.
      • Raman G.
      • Berkman N.D.
      • Grant M.
      • et al.
      Development quality criteria to evaluate nontherapeutic studies of incidence, prevalence, or risk factors of chronic diseases: pilot study of new checklists.
      ,
      • Silva L.C.
      • Ordunez P.
      • Paz Rodriguez M.
      • Robles S.
      A tool for assessing the usefulness of prevalence studies done for surveillance purposes: the example of hypertension.
      ,
      • Munn Z.
      • Moola S.
      • Riitano D.
      • Lisy K.
      The development of a critical appraisal tool for use in systematic reviews addressing questions of prevalence.
      ,
      • Munn Z.
      • Moola S.
      • Lisy K.
      • Riitano D.
      • Tufanaru C.
      Methodological guidance for systematic reviews of observational epidemiological studies reporting prevalence and cumulative incidence data.
      ,
      Academy of Nutrition and Dietetics
      Evidence analysis manual: steps in the academy evidence analysis process.
      ,
      • Avis M.
      Reading research critically. II. An introduction to appraisal: assessing the evidence.
      ,
      • Downes M.J.
      • Brennan M.L.
      • Williams H.C.
      • Dean R.S.
      Development of a critical appraisal tool to assess the quality of cross-sectional studies (AXIS).
      ,
      • Berra S.
      • Elorza-Ricart J.M.
      • Estrada M.D.
      • Sanchez E.
      [A tool (corrected) for the critical appraisal of epidemiological cross-sectional studies].
      ,

      The University of Manchester. Centre for occupational and environmental health (COEH) - critical appraisal.http://research.bmh.manchester.ac.uk/epidemiology/COEH/undergraduate/specialstudymodules/criticalappraisal/Accessed 1 August 2020

      ,
      • Downs S.H.
      • Black N.
      The feasibility of creating a checklist for the assessment of the methodological quality both of randomised and non-randomised studies of health care interventions.
      ,
      • DuRant R.H.
      Checklist for the evaluation of research articles.
      ,
      • Fowkes F.G.
      • Fulton P.M.
      Critical appraisal of published research: introductory guidelines.
      ,
      • Gardner M.J.
      • Machin D.
      • Campbell M.J.
      Use of check lists in assessing the statistical content of medical studies.
      ,
      • Glynn L.
      A critical appraisal tool for library and information research.
      ,
      • Kmet L.S.
      • Lee R.C.
      Standard quality assessment criteria for evaluating primary research papers from a variety of fieldsAHFMRHTA initiative20040213. HTA Initiative.
      ,
      • Law M.
      • Stewart D.
      • Pollock N.
      • Letts L.
      • Bosch J.
      • Westmorland M.
      ,
      • Margetts B.
      • Vorster H.
      • Venter C.
      Evidence-based nutrition-review of nutritional epidemiological studies.
      ,
      • Slim K.
      • Nini E.
      • Forestier D.
      • Kwiatkowski F.
      • Panis Y.
      • Chipponi J.
      Methodological index for non-randomized studies (minors): development and validation of a new instrument.
      ,
      • Hong Q.N.
      • Pluye P.
      • Fàbregues S.
      • Bartlett G.
      • Boardman F.
      • Cargo M.
      • et al.
      Mixed methods appraisal tool (MMAT), version 2018.
      ,
      • Wells G.
      • Shea B.
      • O’Connell D.
      • Peterson J.
      • Welch V.
      • Losos M.
      • et al.
      Newcastle-Ottawa quality assessment scale cohort studies.
      ,
      NIH
      Quality Assessment tool for observational cohort and cross-sectional studies.
      , ,
      • Wong W.C.W.
      • Cheung C.S.K.
      • Hart G.J.
      Development of a quality assessment tool for systematic reviews of observational studies (QATSO) of HIV prevalence in men having sex with men and associated risk behaviours.
      ,
      • Viswanathan M.
      • Berkman N.D.
      Development of the RTI item bank on risk of bias and precision of observational studies.
      ,
      • Viswanathan M.
      • Berkman N.D.
      • Dryden D.M.
      • Hartling L.
      ahrq methods for effective health care. assessing risk of bias and confounding in observational studies of interventions or exposures: further development of the RTI item bank.
      ,
      Specialist Unit for Review Evidence (SURE)
      Questions to assist with the critical appraisal of cross-sectional studies.
      ]. Fig. 2 presents the flowchart of study selection. The list of full texts excluded with reason is available in Additional file 2.

      3.2 Tools specific for prevalence studies

      Out of the 30 tools, eight (26.7%) were specifically designed to appraise prevalence studies [
      • Al-Jader L.N.
      • Newcombe R.G.
      • Hayes S.
      • Murray A.
      • Layzell J.
      • Harper P.S.
      Developing a quality scoring system for epidemiological surveys of genetic disorders.
      ,
      • Boyle M.H.
      Guidelines for evaluating prevalence studies.
      ,
      • Giannakopoulos N.N.
      • Rammelsberg P.
      • Eberhard L.
      • Schmitter M.
      A new instrument for assessing the quality of studies on prevalence.
      ,
      • Hoy D.
      • Brooks P.
      • Woolf A.
      • Blyth F.
      • March L.
      • Bain C.
      • et al.
      Assessing risk of bias in prevalence studies: modification of an existing tool and evidence of interrater agreement.
      ,
      • Loney P.L.
      • Chambers L.W.
      • Bennett K.J.
      • Roberts J.G.
      • Stratford P.W.
      Critical appraisal of the health research literature: prevalence or incidence of a health problem.
      ,
      • Shamliyan T.A.
      • Kane R.L.
      • Ansari M.T.
      • Raman G.
      • Berkman N.D.
      • Grant M.
      • et al.
      Development quality criteria to evaluate nontherapeutic studies of incidence, prevalence, or risk factors of chronic diseases: pilot study of new checklists.
      ,
      • Silva L.C.
      • Ordunez P.
      • Paz Rodriguez M.
      • Robles S.
      A tool for assessing the usefulness of prevalence studies done for surveillance purposes: the example of hypertension.
      ,
      • Munn Z.
      • Moola S.
      • Riitano D.
      • Lisy K.
      The development of a critical appraisal tool for use in systematic reviews addressing questions of prevalence.
      ,
      • Munn Z.
      • Moola S.
      • Lisy K.
      • Riitano D.
      • Tufanaru C.
      Methodological guidance for systematic reviews of observational epidemiological studies reporting prevalence and cumulative incidence data.
      ]. Table 1 summarizes the main characteristics of these tools. Among these tools, seven (87.5%) were new tools [
      • Al-Jader L.N.
      • Newcombe R.G.
      • Hayes S.
      • Murray A.
      • Layzell J.
      • Harper P.S.
      Developing a quality scoring system for epidemiological surveys of genetic disorders.
      ,
      • Boyle M.H.
      Guidelines for evaluating prevalence studies.
      ,
      • Giannakopoulos N.N.
      • Rammelsberg P.
      • Eberhard L.
      • Schmitter M.
      A new instrument for assessing the quality of studies on prevalence.
      ,
      • Loney P.L.
      • Chambers L.W.
      • Bennett K.J.
      • Roberts J.G.
      • Stratford P.W.
      Critical appraisal of the health research literature: prevalence or incidence of a health problem.
      ,
      • Shamliyan T.A.
      • Kane R.L.
      • Ansari M.T.
      • Raman G.
      • Berkman N.D.
      • Grant M.
      • et al.
      Development quality criteria to evaluate nontherapeutic studies of incidence, prevalence, or risk factors of chronic diseases: pilot study of new checklists.
      ,
      • Silva L.C.
      • Ordunez P.
      • Paz Rodriguez M.
      • Robles S.
      A tool for assessing the usefulness of prevalence studies done for surveillance purposes: the example of hypertension.
      ,
      • Munn Z.
      • Moola S.
      • Riitano D.
      • Lisy K.
      The development of a critical appraisal tool for use in systematic reviews addressing questions of prevalence.
      ,
      • Munn Z.
      • Moola S.
      • Lisy K.
      • Riitano D.
      • Tufanaru C.
      Methodological guidance for systematic reviews of observational epidemiological studies reporting prevalence and cumulative incidence data.
      ], and one (12.5%) was an adaptation of an existing instrument [
      • Hoy D.
      • Brooks P.
      • Woolf A.
      • Blyth F.
      • March L.
      • Bain C.
      • et al.
      Assessing risk of bias in prevalence studies: modification of an existing tool and evidence of interrater agreement.
      ]. Four tools (50.0%) were developed to assess studies of prevalence of any clinical condition [
      • Giannakopoulos N.N.
      • Rammelsberg P.
      • Eberhard L.
      • Schmitter M.
      A new instrument for assessing the quality of studies on prevalence.
      ,
      • Hoy D.
      • Brooks P.
      • Woolf A.
      • Blyth F.
      • March L.
      • Bain C.
      • et al.
      Assessing risk of bias in prevalence studies: modification of an existing tool and evidence of interrater agreement.
      ,
      • Loney P.L.
      • Chambers L.W.
      • Bennett K.J.
      • Roberts J.G.
      • Stratford P.W.
      Critical appraisal of the health research literature: prevalence or incidence of a health problem.
      ,
      • Munn Z.
      • Moola S.
      • Riitano D.
      • Lisy K.
      The development of a critical appraisal tool for use in systematic reviews addressing questions of prevalence.
      ,
      • Munn Z.
      • Moola S.
      • Lisy K.
      • Riitano D.
      • Tufanaru C.
      Methodological guidance for systematic reviews of observational epidemiological studies reporting prevalence and cumulative incidence data.
      ], whereas the other four tools (50.0%) were developed to appraise prevalence studies of specific medical fields [
      • Al-Jader L.N.
      • Newcombe R.G.
      • Hayes S.
      • Murray A.
      • Layzell J.
      • Harper P.S.
      Developing a quality scoring system for epidemiological surveys of genetic disorders.
      ,
      • Boyle M.H.
      Guidelines for evaluating prevalence studies.
      ,
      • Shamliyan T.A.
      • Kane R.L.
      • Ansari M.T.
      • Raman G.
      • Berkman N.D.
      • Grant M.
      • et al.
      Development quality criteria to evaluate nontherapeutic studies of incidence, prevalence, or risk factors of chronic diseases: pilot study of new checklists.
      ,
      • Silva L.C.
      • Ordunez P.
      • Paz Rodriguez M.
      • Robles S.
      A tool for assessing the usefulness of prevalence studies done for surveillance purposes: the example of hypertension.
      ]; however, with some adaptations, they could all be applied to any clinical condition. The process of development of all instruments included search and review of relevant criteria, piloting test(s), and adjustments for the final version. The median number of questions in the tools was 10, ranging from seven to 32. Four tools (50.0%) were scales, with numeric results [
      • Al-Jader L.N.
      • Newcombe R.G.
      • Hayes S.
      • Murray A.
      • Layzell J.
      • Harper P.S.
      Developing a quality scoring system for epidemiological surveys of genetic disorders.
      ,
      • Giannakopoulos N.N.
      • Rammelsberg P.
      • Eberhard L.
      • Schmitter M.
      A new instrument for assessing the quality of studies on prevalence.
      ,
      • Loney P.L.
      • Chambers L.W.
      • Bennett K.J.
      • Roberts J.G.
      • Stratford P.W.
      Critical appraisal of the health research literature: prevalence or incidence of a health problem.
      ,
      • Silva L.C.
      • Ordunez P.
      • Paz Rodriguez M.
      • Robles S.
      A tool for assessing the usefulness of prevalence studies done for surveillance purposes: the example of hypertension.
      ], and four (50.0%) were descriptive checklists [
      • Boyle M.H.
      Guidelines for evaluating prevalence studies.
      ,
      • Hoy D.
      • Brooks P.
      • Woolf A.
      • Blyth F.
      • March L.
      • Bain C.
      • et al.
      Assessing risk of bias in prevalence studies: modification of an existing tool and evidence of interrater agreement.
      ,
      • Shamliyan T.A.
      • Kane R.L.
      • Ansari M.T.
      • Raman G.
      • Berkman N.D.
      • Grant M.
      • et al.
      Development quality criteria to evaluate nontherapeutic studies of incidence, prevalence, or risk factors of chronic diseases: pilot study of new checklists.
      ,
      • Munn Z.
      • Moola S.
      • Riitano D.
      • Lisy K.
      The development of a critical appraisal tool for use in systematic reviews addressing questions of prevalence.
      ,
      • Munn Z.
      • Moola S.
      • Lisy K.
      • Riitano D.
      • Tufanaru C.
      Methodological guidance for systematic reviews of observational epidemiological studies reporting prevalence and cumulative incidence data.
      ]. Among the scales, only one suggested cutoff values to define the overall quality of the study [
      • Giannakopoulos N.N.
      • Rammelsberg P.
      • Eberhard L.
      • Schmitter M.
      A new instrument for assessing the quality of studies on prevalence.
      ]; among the checklists, two had an overall appraisal question, but they were answered based on rater's judgment, without guidance of how to consider the previous questions to define a summary assessment [
      • Hoy D.
      • Brooks P.
      • Woolf A.
      • Blyth F.
      • March L.
      • Bain C.
      • et al.
      Assessing risk of bias in prevalence studies: modification of an existing tool and evidence of interrater agreement.
      ,
      • Munn Z.
      • Moola S.
      • Riitano D.
      • Lisy K.
      The development of a critical appraisal tool for use in systematic reviews addressing questions of prevalence.
      ,
      • Munn Z.
      • Moola S.
      • Lisy K.
      • Riitano D.
      • Tufanaru C.
      Methodological guidance for systematic reviews of observational epidemiological studies reporting prevalence and cumulative incidence data.
      ].
      Table 1Main characteristics of tools specifically designed to appraise prevalence studies
      Instrument (name of the instrument or author, year)Context of development (clinical condition)Process of developmentStructureSummary and reporting of results
      Al-Jader et al., 2002 [
      • Al-Jader L.N.
      • Newcombe R.G.
      • Hayes S.
      • Murray A.
      • Layzell J.
      • Harper P.S.
      Developing a quality scoring system for epidemiological surveys of genetic disorders.
      ]
      Genetic disorders.First version of the tool; pilot test with multidisciplinary assessors to evaluate reproducibility and feasibility; final version of the tool and test for inter-rater agreement.Seven questions, with different answer options; each answer option with an associated score.Maximum score: 100 points.

      No cutoff point defined.
      Boyle, 1998 [
      • Boyle M.H.
      Guidelines for evaluating prevalence studies.
      ]
      Psychiatric disorders on general population settings.NRTen questions, split in three sections. No predefined answer options.No overall summary.

      Descriptive reporting of results.
      Giannakapoulos et al., 2012 [
      • Giannakopoulos N.N.
      • Rammelsberg P.
      • Eberhard L.
      • Schmitter M.
      A new instrument for assessing the quality of studies on prevalence.
      ]
      Any clinical condition.Search for criteria to define a high-quality study of prevalence; development of the first version; pilot tests to determine inter-rater agreement and reliability; the final version of the tool.Eleven questions, split in three sections, plus a question about ethics. Each question with two or three answer options; each answer option with an associated score.Maximum score: 19 points. Studies are classified in accordance with their total score as poor (0–4), moderate (5–9), good (10–14), or outstanding (15–19).
      Hoy et al., 2012 [
      • Hoy D.
      • Brooks P.
      • Woolf A.
      • Blyth F.
      • March L.
      • Bain C.
      • et al.
      Assessing risk of bias in prevalence studies: modification of an existing tool and evidence of interrater agreement.
      ]
      Any clinical condition.Search for instruments; definition of important criteria to be assessed and creation of the draft tool; pilot tests with professionals; assessment of inter-rater agreement, ease of use, timeliness; the final version of the tool.Ten questions with two standard answer options (high risk of bias/low risk of bias).Question for overall appraisal with three answer options (low risk of bias/moderate risk of bias/high risk of bias), based on rater's judgment.
      Loney et al., 1998 [
      • Loney P.L.
      • Chambers L.W.
      • Bennett K.J.
      • Roberts J.G.
      • Stratford P.W.
      Critical appraisal of the health research literature: prevalence or incidence of a health problem.
      ]
      Any clinical condition.Review of important criteria; development of the tool; pilot test in prevalence studies of dementia.Eight statements, one point for each criterion achieved.Maximum score: eight points.

      No cutoff point defined.
      MORE, 2010 [
      • Shamliyan T.A.
      • Kane R.L.
      • Ansari M.T.
      • Raman G.
      • Berkman N.D.
      • Grant M.
      • et al.
      Development quality criteria to evaluate nontherapeutic studies of incidence, prevalence, or risk factors of chronic diseases: pilot study of new checklists.
      ]
      Chronic conditions.Systematic search for instruments to assess prevalence and incidence studies; selection of important criteria; development of the first version; pilot test with experts to assess face validity, inter-rater agreement, and reliability; the final version of the tool.Thirty-two questions, with different answer options; each answer option is classified as ‘minor flaw’, ‘major flaw’, or ‘poor reporting’.No overall summary

      Descriptive reporting of results.
      Silva et al., 2001 [
      • Silva L.C.
      • Ordunez P.
      • Paz Rodriguez M.
      • Robles S.
      A tool for assessing the usefulness of prevalence studies done for surveillance purposes: the example of hypertension.
      ]
      Risk factors of chronic diseases.NRNineteen questions, split in three sections. Two or three answer options, with an associated score.Maximum score: 100 points.

      No cutoff point defined.
      The Joanna Briggs Institute Prevalence Critical Appraisal Tool, 2014 [
      • Munn Z.
      • Moola S.
      • Riitano D.
      • Lisy K.
      The development of a critical appraisal tool for use in systematic reviews addressing questions of prevalence.
      ,
      • Munn Z.
      • Moola S.
      • Lisy K.
      • Riitano D.
      • Tufanaru C.
      Methodological guidance for systematic reviews of observational epidemiological studies reporting prevalence and cumulative incidence data.
      ]
      Any clinical condition.Systematic search for instruments to assess prevalence studies; review and selection of applicable criteria; development of the draft tool; pilot tests with professionals to assess face validity, applicability, acceptability, timeliness, and ease of use; the final version of the tool.Nine questions with four standard answer options (yes/no/unclear/not applicable).
      Ten questions in previous versions.
      Question for overall appraisal with three answer options (include/exclude/seek further info), based on rater's judgment.
      Abbreviation: NR, not reported.
      a Ten questions in previous versions.
      Regarding the domains assessed, seven tools (87.5%) covered all key domains [
      • Boyle M.H.
      Guidelines for evaluating prevalence studies.
      ,
      • Giannakopoulos N.N.
      • Rammelsberg P.
      • Eberhard L.
      • Schmitter M.
      A new instrument for assessing the quality of studies on prevalence.
      ,
      • Hoy D.
      • Brooks P.
      • Woolf A.
      • Blyth F.
      • March L.
      • Bain C.
      • et al.
      Assessing risk of bias in prevalence studies: modification of an existing tool and evidence of interrater agreement.
      ,
      • Loney P.L.
      • Chambers L.W.
      • Bennett K.J.
      • Roberts J.G.
      • Stratford P.W.
      Critical appraisal of the health research literature: prevalence or incidence of a health problem.
      ,
      • Shamliyan T.A.
      • Kane R.L.
      • Ansari M.T.
      • Raman G.
      • Berkman N.D.
      • Grant M.
      • et al.
      Development quality criteria to evaluate nontherapeutic studies of incidence, prevalence, or risk factors of chronic diseases: pilot study of new checklists.
      ,
      • Silva L.C.
      • Ordunez P.
      • Paz Rodriguez M.
      • Robles S.
      A tool for assessing the usefulness of prevalence studies done for surveillance purposes: the example of hypertension.
      ,
      • Munn Z.
      • Moola S.
      • Riitano D.
      • Lisy K.
      The development of a critical appraisal tool for use in systematic reviews addressing questions of prevalence.
      ,
      • Munn Z.
      • Moola S.
      • Lisy K.
      • Riitano D.
      • Tufanaru C.
      Methodological guidance for systematic reviews of observational epidemiological studies reporting prevalence and cumulative incidence data.
      ]; however, there was variability regarding the items assessed by each tool. Table 2 describes the items of each tool, classified by domains. In the domain “population and setting”, the main items assessed were “appropriate sampling” (seven tools, 87.5%), “appropriate response rate” (five tools, 62.5%), and “representative sample” (four tools, 50.0%). In the domain “condition measurement”, the main items assessed were “valid measurement of condition” (six tools, 75.0%), “standard measurement of condition” (six tools, 75.0%), and “reliable measurement of condition” (five tools, 62.5%). In the domain “statistics”, the main items assessed were “precision of estimate” (six tools, 75.0%), “data analysis considering sampling” (three tools, 37.5%), “appropriate sample size” (two tools, 25.0%), and “subgroup analysis” (two tools, 25.0%). Overall, these tools provided 25 unique items related to the assessment of quality of prevalence studies, six related to “population and setting”, nine related to “condition measurement”, and 10 related to “statistics”. In addition, 24 items were classified as “other”, mainly related to reporting (Table 2).
      Table 2Items assessed by tools specifically designed to appraise prevalence studies, classified by domains
      The full set of question for every tool is presented in Additional file 5.
      Instrument (name of the instrument or author, year)Domain
      Population and settingCondition measurementStatisticsOther
      Al-Jader et al., 2002 [
      • Al-Jader L.N.
      • Newcombe R.G.
      • Hayes S.
      • Murray A.
      • Layzell J.
      • Harper P.S.
      Developing a quality scoring system for epidemiological surveys of genetic disorders.
      ]
      Representative sample, ethnic characteristics of population source, appropriate size of population source
      Tool specific for studies of prevalence of genetic conditions.
      Precision of estimatesDescription of condition of interest, reporting of year of conduction of studies, reporting of size of population source
      Boyle, 1998 [
      • Boyle M.H.
      Guidelines for evaluating prevalence studies.
      ]
      Representative sample, appropriate samplingReliable and valid measurement of conditionPrecision of estimates, data analysis considering samplingDescription of target population, standard data collection
      Giannakapoulos et al., 2012 [
      • Giannakopoulos N.N.
      • Rammelsberg P.
      • Eberhard L.
      • Schmitter M.
      A new instrument for assessing the quality of studies on prevalence.
      ]
      Appropriate sampling and response rateReliable, standard, and valid measurement of conditionPrecision of estimates, data analysis considering response rate and special featuresDescription of target population, ethics
      Hoy et al., 2012 [
      • Hoy D.
      • Brooks P.
      • Woolf A.
      • Blyth F.
      • March L.
      • Bain C.
      • et al.
      Assessing risk of bias in prevalence studies: modification of an existing tool and evidence of interrater agreement.
      ]
      Representative sample, appropriate sampling, and appropriate response rateAppropriate definition of condition, reliable, standard, and valid measurement of condition, and appropriate length of prevalence periodAppropriate numerator and denominator parametersAppropriate data collection
      Loney et al., 1998 [
      • Loney P.L.
      • Chambers L.W.
      • Bennett K.J.
      • Roberts J.G.
      • Stratford P.W.
      Critical appraisal of the health research literature: prevalence or incidence of a health problem.
      ]
      Appropriate sampling and appropriate response rateAppropriate, standard, and unbiased measurement of conditionAppropriate sample size, precision of estimatesAppropriate study design, description of participants, setting, and nonresponders
      MORE, 2010 [
      • Shamliyan T.A.
      • Kane R.L.
      • Ansari M.T.
      • Raman G.
      • Berkman N.D.
      • Grant M.
      • et al.
      Development quality criteria to evaluate nontherapeutic studies of incidence, prevalence, or risk factors of chronic diseases: pilot study of new checklists.
      ]
      Appropriate sampling and appropriate response rateAppropriate, reliable, standard, and valid measurement of condition, assessment of disease severity and frequency of symptoms, type of prevalence estimate (point or period), and appropriate length of prevalence periodPrecision of estimates, appropriate exclusion from analysis, data analysis considering sampling, subgroup analysis, and adjustment of estimatesReporting of study design, description of study objectives, reporting of inclusion flowchart, description, and role funding, and reporting of conflict of interest, ethics
      Silva et al., 2001 [
      • Silva L.C.
      • Ordunez P.
      • Paz Rodriguez M.
      • Robles S.
      A tool for assessing the usefulness of prevalence studies done for surveillance purposes: the example of hypertension.
      ]
      Appropriate sample source and appropriate samplingAppropriate definition of condition, standard and valid measurement of conditionPrecision of estimates, subgroup analysis, data analysis considering samplingDescription of study objectives and sampling frame, quality control of data, applicability and generalizability of results
      The Joanna Briggs Institute Prevalence Critical Appraisal Tool, 2014 [
      • Munn Z.
      • Moola S.
      • Riitano D.
      • Lisy K.
      The development of a critical appraisal tool for use in systematic reviews addressing questions of prevalence.
      ,
      • Munn Z.
      • Moola S.
      • Lisy K.
      • Riitano D.
      • Tufanaru C.
      Methodological guidance for systematic reviews of observational epidemiological studies reporting prevalence and cumulative incidence data.
      ]
      Representative sample, appropriate sampling, and appropriate response rateReliable, standard, and valid measurement of conditionAppropriate sample size, appropriate statistical analysis, data analysis considering response rateDescription of participants and setting, objective criteria for subgroup definitions
      a The full set of question for every tool is presented in Additional file 5.
      b Tool specific for studies of prevalence of genetic conditions.

      3.3 Tools adapted for prevalence studies

      Among the 30 included tools, 22 (73.3%) were not specific for prevalence studies. The main characteristics of these tools and the items and domains assessed by them are presented in Additional file 3 and 4, respectively. These tools provided six unique additional items for the domain “population and setting”, seven items related to “condition measurement”, and four items related to “statistics”. Moreover, these tools provided 53 items classified under the domain “other” (Additional file 4).

      3.4 Items

      We identified 710 questions/statements from the included tools that were compiled into 119 different items.
      We identified 42 unique items classified under the domains “population and setting” (12 items), “condition measurement” (16 items), and “statistics” (14 items); of those, 25 (59.5%) were identified in the eight specific tools and 17 (40.5%) were identified in the additional 22 nonspecific tools. Table 3 summarized the items assessed in each domain among all tools.
      Table 3Unique items identified among all included tools and classified into a key domain
      Population and setting (n = 12)Condition measurement (n = 16)Statistics (n = 14)
      • Appropriate sample
        Items from nonspecific tools only.
        (seven tools)
      • Unbiased sample
        Items from nonspecific tools only.
        (one tool)
      • Representative sample (14 tools)
      • Appropriate sample source (two tools)
      • Appropriate size of population source (one tool)
      • Ethnic characteristics of population source (one tool)
      • Appropriate sampling (15 tools)
      • Random sampling
        Items from nonspecific tools only.
        (one tool)
      • Standard selection of participants
        Items from nonspecific tools only.
        (two tools)
      • Participation rate of eligible persons
        Items from nonspecific tools only.
        (one tool)
      • Appropriate response rate (19 tools)
      • Assessment of nonresponders
        Items from nonspecific tools only.
        (one tool)
      • Type of prevalence estimate (point or period) (one tool)
      • Appropriate length of prevalence period (two tools)
      • Appropriate definition of condition (two tools)
      • Appropriate measurement of condition (11 tools)
      • Accurate measurement of condition
        Items from nonspecific tools only.
        (one tool)
      • Precise measurement of condition
        Items from nonspecific tools only.
        (one tool)
      • Quality control of measurement methods
        Items from nonspecific tools only.
        (two tools)
      • Valid measurement of condition (21 tools)
      • Reliable measurement of condition (15 tools)
      • Standard measurement of condition (10 tools)
      • Unbiased measurement of condition (three tools)
      • Reproducible measurement of condition
        Items from nonspecific tools only.
        (two tools)
      • Assessment of disease severity and frequency of symptoms (one tool)
      • Data collection performed by investigators unrelated to patients
        Items from nonspecific tools only.
        (one tool)
      • Face validity
        Items from nonspecific tools only.
        (one tool)
      • Selective outcome reporting
        Items from nonspecific tools only.
        (one tool)
      • Sample size estimation
        Items from nonspecific tools only.
        (10 tools)
      • Appropriate sample size (seven tools)
      • Appropriate statistical analysis (13 tools)
      • Appropriate numerator and denominator parameters (one tool)
      • Appropriate exclusion from analysis (one tool)
      • Adjustment of estimates (one tool)
      • Data analysis considering sampling (three tools)
      • Data analysis considering the response rate (seven tools)
      • Data analysis considering special features (one tool)
      • Missing data handling
        Items from nonspecific tools only.
        (one tool)
      • Random error
        Items from nonspecific tools only.
        (three tools)
      • Precision of estimate (11 tools)
      • Subgroup analysis (five tools)
      • Data fishing
        Items from nonspecific tools only.
        (two tools)
      a Items from nonspecific tools only.
      In the domain “other”, we identified 77 unique items; of those, 24 (31.2%) were identified in the eight specific tools and 53 (68.8%) were identified in the additional 22 nonspecific tools. We classified the items in the domain “other” into three categories: “manuscript writing and reporting”, “study protocol and methods”, and “nonclassified”. These categories were defined after data extraction, based on our findings. Table 4 presents the items classified as “other” stratified by these categories.
      Table 4Items classified as ‘other’, categorized into three subgroups
      Manuscript writing and reporting (n = 48)Study protocol and methods (n = 12)Nonclassified (n = 17)
      • Clear reporting of authors and affiliations
        Items from nonspecific tools only.
        (one tool)
      • Appropriate title
        Items from nonspecific tools only.
        (one tool)
      • Appropriate abstract
        Items from nonspecific tools only.
        (one tool)
      • Study justified by literature review
        Items from nonspecific tools only.
        (one tool)
      • Description of the problem
        Items from nonspecific tools only.
        (one tool)
      • Theoretical framework
        Items from nonspecific tools only.
        (one tool)
      • Clear hypothesis
        Items from nonspecific tools only.
        (one tool)
      • Clear study questions
        Items from nonspecific tools only.
        (eight tools)
      • Description of study objectives (13 tools)
      • Description of condition of interest (10 tools)
      • Description of target population (seven tools)
      • Description of the setting (six tools)
      • Reporting of the study design (five tools)
      • Description of methods
        Items from nonspecific tools only.
        (three tools)
      • Reporting of the size of the population source (one tool)
      • Description of the sampling frame (five tools)
      • Description of eligibility criteria
        Items from nonspecific tools only.
        (seven tools)
      • Reporting of the year of conduction of study (one tool)
      • Description of statistical analysis (seven tools)
      • Appropriate data reporting
        Items from nonspecific tools only.
        (three tools)
      • Appropriate reporting of results
        Items from nonspecific tools only.
        (10 tools)
      • Reporting of inclusion flowchart (three tools)
      • Reporting of sample size (one tool)
      • Reporting of the response rate
        Items from nonspecific tools only.
        (three tools)
      • Description of nonresponders (four tools)
      • Description of participants (10 tools)
      • Reporting of data collection procedures
        Items from nonspecific tools only.
        (two tools)
      • Clear description of data sources
        Items from nonspecific tools only.
        (one tool)
      • Description of missing data
        Items from nonspecific tools only.
        (two tools)
      • Reporting of adjusted estimates
        Items from nonspecific tools only.
        (one tool)
      • Reporting of statistical significance
        Items from nonspecific tools only.
        (two tools)
      • Reporting of clinical significance
        Items from nonspecific tools only.
        (two tools)
      • Reporting of discussion
        Items from nonspecific tools only.
        (one tool)
      • Appropriate discussion
        Items from nonspecific tools only.
        (three tools)
      • Discussion based on results
        Items from nonspecific tools only.
        (one tool)
      • Reporting of all possible interpretation of results
        Items from nonspecific tools only.
        (one tool)
      • Discussion of bias
        Items from nonspecific tools only.
        (two tools)
      • Discussion of limitations
        Items from nonspecific tools only.
        (five tools)
      • Discussion of strengths
        Items from nonspecific tools only.
        (one tool)
      • Comparison of results with the existing literature
        Items from nonspecific tools only.
        (three tools)
      • Description of study conclusions
        Items from nonspecific tools only.
        (one tool)
      • Appropriate conclusions
        Items from nonspecific tools only.
        (two tools)
      • Conclusion based on results
        Items from nonspecific tools only.
        (10 tools)
      • Reporting of an additional information source
        Items from nonspecific tools only.
        (one tool)
      • Description of funding (two tools)
      • Reporting of conflict of interest (four tools)
      • Clear references
        Items from nonspecific tools only.
        (one tool)
      • Recommendations for future research
        Items from nonspecific tools only.
        (two tools)
      • Specific objectives
        Items from nonspecific tools only.
        (two tools)
      • Study protocol
        Items from nonspecific tools only.
        (one tool)
      • A priori statistical analysis plan
        Items from nonspecific tools only.
        (one tool)
      • Appropriate study design (11 tools)
      • Appropriate review of the existing literature
        Items from nonspecific tools only.
        (two tools)
      • Appropriate methods
        Items from nonspecific tools only.
        (three tools)
      • Appropriate data collection (two tools)
      • Standard data collection (one tool)
      • Consideration of important variables
        Items from nonspecific tools only.
        (one tool)
      • Consideration of privacy and sensitivity of condition
        Items from nonspecific tools only.
        (one tool)
      • Objective criteria for subgroup definitions (one tool)
      • Importance of study
        Items from nonspecific tools only.
        (three tools)
      • Quality control of data (one tool)
      • Relevance of the research question
        Items from nonspecific tools only.
        (one tool)
      • Relevance of outcomes
        Items from nonspecific tools only.
        (one tool)
      • Identification of bias
        Items from nonspecific tools only.
        (one tool)
      • Consistent results
        Items from nonspecific tools only.
        (three tools)
      • Believable results
        Items from nonspecific tools only.
        (one tool)
      • Conclusion plausible
        Items from nonspecific tools only.
        (two tools)
      • Possible alternative conclusions
        Items from nonspecific tools only.
        (one tool)
      • Relevance of conclusions
        Items from nonspecific tools only.
        (one tool)
      • Applicability of results (one tool)
      • Generalizability of results (six tools)
      • Ethics (six tools)
      • Effect of conflict of interest
        Items from nonspecific tools only.
        (two tools)
      • Role of funding (one tool)
      • Bias due to funding
        Items from nonspecific tools only.
        (one tool)
      • Reader's interpretation of study
        Items from nonspecific tools only.
        (three tools)
      • Other
        Items from nonspecific tools only.
        (one tool)
      a Items from nonspecific tools only.

      4. Discussion

      In this systematic review, we identified, summarized, and compared 30 instruments used for the quality assessment of prevalence studies. Our results, similar to what was found in other reviews, show that there is great variability among tools and there is no consensus about which domains should be assessed in prevalence studies [
      • Sanderson S.
      • Tatt I.D.
      • Higgins J.P.
      Tools for assessing quality and susceptibility to bias in observational studies in epidemiology: a systematic review and annotated bibliography.
      ,
      • Jarde A.
      • Losilla J.M.
      • Vives J.
      Methodological quality assessment tools of non-experimental studies: a systematic review.
      ]. We classified all questions or statements into items and domains, creating a comprehensive set of 119 items useful for the assessment of prevalence studies.
      Not all domains were covered by all tools, and even when they were covered, they were not always properly assessed. Some tools did not consider important aspects inside each domain, such as representativeness of sample, estimation of sample size, and appropriate measurement of condition, and there was an overlap among questions in the same instrument, which may lead to penalization of the same study for the same reason more than once. Moreover, many instruments assessed not only risk of bias but also reporting and manuscript writing. It is important to distinguish between these two concepts as poor reporting is not a reflection on the quality of a study or whether the results from a study are at risk of bias.
      We conducted a broad search, using important databases and including alternative data sources. Our search was very sensitive to identify tools specifically designed for prevalence studies, but we probably have not included all instruments that could be adapted for this purpose. This could be a limitation of our study; however, we believe our results are representative of the items and domains used to appraise prevalence studies because we probably achieved a saturation of items, and this work is the most comprehensive overview of tools to assess prevalence studies to date. Another possible limitation of our review is that data abstraction and classification of questions into items and domains required judgment, which can lead to different decisions by different assessors. We tried to overcome this by conducting the classification independently by two reviewers with the assistance of third reviewers in case of discrepancies. In addition, our decision-making was informed by a protocol which reduced the chance of making ad hoc, subjective decisions during the conduct of our review. Moreover, to enhance transparency of the process, our judgments regarding the classification of questions and statements into items and domains are available in Additional file 5.
      The main objective of this study was to map the literature to identify items used to assess the risk of bias of prevalence studies, generating a comprehensive bank of items. Some of the items identified are similar and there may be overlap among them in terms of their broad definition. However, we do believe that subtle differences in terminology and nomenclature are important in the contextualization of the tools in which they are derived. This granular approach is justified in this case as this will facilitate the future development of a new risk of bias assessment tool. As an example of the importance of identifying subtle differences between similar items in tools, we attempt in the following to clarify the differences between valid, reliable, reproducible, and unbiased measurement of the condition (which some readers may have considered synonymous terms previously):
      • Valid measurement of the condition: the measurement of the condition is performed with methods that actually measures or detects what it is supposed to measure.
      • Reliable measurement of the condition: this is related to the consistency of a measure. A highly reliable measure produces similar results under similar conditions, so all things being equal, repeated testing should produce similar results.
      • Reproducible measurement of the condition: the measurement of the condition is performed using methods capable of being reproduced at a different time or place and by different people
      • Unbiased measurement of the condition: measurement of the condition free of systematic errors that could deviate the results from the truth.
      Therefore, even though the differences among items are subtle, we do believe they are important to inform the development of a new tool.
      It is not possible to strongly recommend a tool because there is great variability in their content. A new tool, domain based and with broader coverage and applicability is needed. However, among the currently available tools specific for prevalence studies, the Joanna Briggs Institute Prevalence Critical Appraisal Tool has a higher methodologic rigor and addresses what we consider the most important items related to the methodological quality of prevalence studies and may be considered the most appropriate tool [
      • Munn Z.
      • Moola S.
      • Riitano D.
      • Lisy K.
      The development of a critical appraisal tool for use in systematic reviews addressing questions of prevalence.
      ,
      • Munn Z.
      • Moola S.
      • Lisy K.
      • Riitano D.
      • Tufanaru C.
      Methodological guidance for systematic reviews of observational epidemiological studies reporting prevalence and cumulative incidence data.
      ].

      5. Conclusions

      We have now identified a comprehensive set of items and domains that is broader than any of the individual tools included in this review. This data set can be now be used by those interested in the critical appraisal of prevalence studies, by authors of prevalence studies, and to inform the development or update of future tools to critically appraise prevalence studies.

      CRediT authorship contribution statement

      Celina Borges Migliavaca: Formal analysis, Investigation, Visualization, Data curation, Writing - original draft. Cinara Stein: Formal analysis, Investigation, Writing - review & editing. Verônica Colpani: Validation, Project administration, Writing - review & editing. Zachary Munn: Methodology, Writing - review & editing. Maicon Falavigna: Conceptualization, Validation, Writing - review & editing.

      References

        • Fletcher R.
        • Fletcher S.
        • Fletcher G.S.
        Clinical epidemiology: the essentials.
        5th ed. Lippincott Williams & Wilkins, Philadelphia2013
        • Harder T.
        Some notes on critical appraisal of prevalence studies: comment on: "The development of a critical appraisal tool for use in systematic reviews addressing questions of prevalence.
        Int J Health Pol Manag. 2014; 3: 289-290
        • Wagner M.B.
        Medindo a corrêencia da doença: prevalência ou incidência?.
        J Pediatr (Rio J. 1998; 74: 157-162
        • Oxman A.D.
        • Schunemann H.J.
        • Fretheim A.
        Improving the use of research evidence in guideline development: 2. Priority setting.
        Health Res Pol Syst. 2006; 4: 14
        • Rotily M.
        • Roze S.
        What is the impact of disease prevalence upon health technology assessment?.
        Best Pract Res Clin Gastroenterol. 2013; 27: 853-865
        • Borges Migliavaca C.
        • Stein C.
        • Colpani V.
        • et al.
        How are systematic reviews of prevalence conducted? A methodological study.
        BMC Med Res Methodol. 2020; 20: 96https://doi.org/10.1186/s12874-020-00975-3
        • Shamliyan T.
        • Kane R.L.
        • Dickinson S.
        A systematic review of tools used to assess the quality of observational studies that examine incidence or prevalence and risk factors for diseases.
        J Clin Epidemiol. 2010; 63: 1061-1070
        • Sterne J.A.
        • Hernan M.A.
        • Reeves B.C.
        • Savovic J.
        • Berkman N.D.
        • Viswanathan M.
        • et al.
        ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions.
        BMJ. 2016; 355: i4919
        • Whiting P.F.
        • Rutjes A.W.
        • Westwood M.E.
        • Mallett S.
        • Deeks J.J.
        • Reitsma J.B.
        • et al.
        QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies.
        Ann Intern Med. 2011; 155: 529-536
        • Sterne J.A.C.
        • Page M.J.
        • Elbers R.G.
        • Blencowe N.S.
        • Boutron I.
        • Cates C.J.
        • et al.
        RoB 2: a revised tool for assessing risk of bias in randomised trials.
        BMJ. 2019; 366: l4898
        • Al-Jader L.N.
        • Newcombe R.G.
        • Hayes S.
        • Murray A.
        • Layzell J.
        • Harper P.S.
        Developing a quality scoring system for epidemiological surveys of genetic disorders.
        Clin Genet. 2002; 62: 230-234
        • Boyle M.H.
        Guidelines for evaluating prevalence studies.
        Evid Based Ment Health. 1998; 1: 37
        • Giannakopoulos N.N.
        • Rammelsberg P.
        • Eberhard L.
        • Schmitter M.
        A new instrument for assessing the quality of studies on prevalence.
        Clin Oral Invest. 2012; 16: 781-788
        • Hoy D.
        • Brooks P.
        • Woolf A.
        • Blyth F.
        • March L.
        • Bain C.
        • et al.
        Assessing risk of bias in prevalence studies: modification of an existing tool and evidence of interrater agreement.
        J Clin Epidemiol. 2012; 65: 934-939
        • Loney P.L.
        • Chambers L.W.
        • Bennett K.J.
        • Roberts J.G.
        • Stratford P.W.
        Critical appraisal of the health research literature: prevalence or incidence of a health problem.
        Chronic Dis Can. 1998; 19: 170-176
        • Shamliyan T.A.
        • Kane R.L.
        • Ansari M.T.
        • Raman G.
        • Berkman N.D.
        • Grant M.
        • et al.
        Development quality criteria to evaluate nontherapeutic studies of incidence, prevalence, or risk factors of chronic diseases: pilot study of new checklists.
        J Clin Epidemiol. 2011; 64: 637-657
        • Silva L.C.
        • Ordunez P.
        • Paz Rodriguez M.
        • Robles S.
        A tool for assessing the usefulness of prevalence studies done for surveillance purposes: the example of hypertension.
        Rev Panam Salud Publica. 2001; 10: 152-160
        • Munn Z.
        • Moola S.
        • Riitano D.
        • Lisy K.
        The development of a critical appraisal tool for use in systematic reviews addressing questions of prevalence.
        Int J Health Pol Manag. 2014; 3: 123-128
        • Munn Z.
        • Moola S.
        • Lisy K.
        • Riitano D.
        • Tufanaru C.
        Methodological guidance for systematic reviews of observational epidemiological studies reporting prevalence and cumulative incidence data.
        Int J Evid Based Healthc. 2015; 13: 147-153
        • Academy of Nutrition and Dietetics
        Evidence analysis manual: steps in the academy evidence analysis process.
        Academy of Nutrition and Dietetics, Chicago2016: 106
        • Avis M.
        Reading research critically. II. An introduction to appraisal: assessing the evidence.
        J Clin Nurs. 1994; 3: 271-277
        • Downes M.J.
        • Brennan M.L.
        • Williams H.C.
        • Dean R.S.
        Development of a critical appraisal tool to assess the quality of cross-sectional studies (AXIS).
        BMJ Open. 2016; 6: 7
        • Berra S.
        • Elorza-Ricart J.M.
        • Estrada M.D.
        • Sanchez E.
        [A tool (corrected) for the critical appraisal of epidemiological cross-sectional studies].
        Gac Sanit. 2008; 22: 492-497
      1. The University of Manchester. Centre for occupational and environmental health (COEH) - critical appraisal.http://research.bmh.manchester.ac.uk/epidemiology/COEH/undergraduate/specialstudymodules/criticalappraisal/Accessed 1 August 2020

        • Downs S.H.
        • Black N.
        The feasibility of creating a checklist for the assessment of the methodological quality both of randomised and non-randomised studies of health care interventions.
        J Epidemiol Community Health. 1998; 52: 377-384
        • DuRant R.H.
        Checklist for the evaluation of research articles.
        J Adolesc Health. 1994; 15: 4-8
        • Fowkes F.G.
        • Fulton P.M.
        Critical appraisal of published research: introductory guidelines.
        BMJ. 1991; 302: 1136-1140
        • Gardner M.J.
        • Machin D.
        • Campbell M.J.
        Use of check lists in assessing the statistical content of medical studies.
        Br Med J (Clin Res Ed). 1986; 292: 810-812
        • Glynn L.
        A critical appraisal tool for library and information research.
        Libr Hi Tech. 2006; 24: 387-399
        • Kmet L.S.
        • Lee R.C.
        Standard quality assessment criteria for evaluating primary research papers from a variety of fieldsAHFMRHTA initiative20040213. HTA Initiative.
        2004: 2
        • Law M.
        • Stewart D.
        • Pollock N.
        • Letts L.
        • Bosch J.
        • Westmorland M.
        Critical review form—quantitative studies. 20. 1998: 13
        • Margetts B.
        • Vorster H.
        • Venter C.
        Evidence-based nutrition-review of nutritional epidemiological studies.
        South Afr J Clin Nutr. 2002; 15: 68-73
        • Slim K.
        • Nini E.
        • Forestier D.
        • Kwiatkowski F.
        • Panis Y.
        • Chipponi J.
        Methodological index for non-randomized studies (minors): development and validation of a new instrument.
        ANZ J Surg. 2003; 73: 712-716
        • Hong Q.N.
        • Pluye P.
        • Fàbregues S.
        • Bartlett G.
        • Boardman F.
        • Cargo M.
        • et al.
        Mixed methods appraisal tool (MMAT), version 2018.
        IC Canadian Intellectual Property Office, Industry Canada, Canada2018
        • Wells G.
        • Shea B.
        • O’Connell D.
        • Peterson J.
        • Welch V.
        • Losos M.
        • et al.
        Newcastle-Ottawa quality assessment scale cohort studies.
        2014 (Available at)
        • NIH
        Quality Assessment tool for observational cohort and cross-sectional studies.
        2018 (Available at)
        • Public Health Wales Observatory
        Critical appraisal checklist: cross sectional study.
        2004 (Available at)
        • Wong W.C.W.
        • Cheung C.S.K.
        • Hart G.J.
        Development of a quality assessment tool for systematic reviews of observational studies (QATSO) of HIV prevalence in men having sex with men and associated risk behaviours.
        Emerging Themes Epidemiol. 2008; 5: 23
        • Viswanathan M.
        • Berkman N.D.
        Development of the RTI item bank on risk of bias and precision of observational studies.
        J Clin Epidemiol. 2012; 65: 163-178
        • Viswanathan M.
        • Berkman N.D.
        • Dryden D.M.
        • Hartling L.
        ahrq methods for effective health care. assessing risk of bias and confounding in observational studies of interventions or exposures: further development of the RTI item bank.
        Agency for Healthcare Research and Quality (US), Rockville, MD2013
        • Specialist Unit for Review Evidence (SURE)
        Questions to assist with the critical appraisal of cross-sectional studies.
        2018 (Available at)
        • Sanderson S.
        • Tatt I.D.
        • Higgins J.P.
        Tools for assessing quality and susceptibility to bias in observational studies in epidemiology: a systematic review and annotated bibliography.
        Int J Epidemiol. 2007; 36: 666-676
        • Jarde A.
        • Losilla J.M.
        • Vives J.
        Methodological quality assessment tools of non-experimental studies: a systematic review.
        An Psicol. 2012; 28: 617-628