Advertisement

Multiple sclerosis disease–related knowledge measurement instruments show mixed performance: a systematic review

  • Marina Gonzalez-del-Rio
    Affiliations
    Neurodegeneration and Neuroinflammation Research Group, Biomedical Research Institute (IDIBGI), Girona, Spain

    Girona Neuroimmunology and Multiple Sclerosis Unit, Neurology Department, Dr Josep Trueta Hospital and Santa Caterina Hospital, Girona-Salt, Spain
    Search for articles by this author
  • Carme Bertran-Noguer
    Affiliations
    Nursing Department, Faculty of Nursing, Universitat de Girona, Girona, Spain

    Health and Health Care Research Group, Universitat de Girona, Girona, Spain
    Search for articles by this author
  • Lluís Ramió-Torrentà
    Affiliations
    Neurodegeneration and Neuroinflammation Research Group, Biomedical Research Institute (IDIBGI), Girona, Spain

    Girona Neuroimmunology and Multiple Sclerosis Unit, Neurology Department, Dr Josep Trueta Hospital and Santa Caterina Hospital, Girona-Salt, Spain

    Medical Sciences Department, University of Girona, Girona, Spain
    Search for articles by this author
  • Edurne Zabaleta-del-Olmo
    Correspondence
    Corresponding author. Gran Via Corts Catalanes, 587 àtic, 08007 Barcelona, Spain. Tel./fax: +34 934 824 105.
    Affiliations
    Nursing Department, Faculty of Nursing, Universitat de Girona, Girona, Spain

    Fundació Institut Universitari per a la recerca a l'Atenció Primària de Salut Jordi Gol i Gurina (IDIAPJGol), Barcelona, Spain

    Gerència Territorial de Barcelona, Institut Català de la Salut, Barcelona, Spain
    Search for articles by this author
Open AccessPublished:May 29, 2022DOI:https://doi.org/10.1016/j.jclinepi.2022.05.020

      Abstract

      Objectives

      This review aimed to summarize the evidence on the measurement properties of available disease-related knowledge measurement instruments in people with multiple sclerosis.

      Study Design and Setting

      We performed a literature search in the MEDLINE (PubMed), CINAHL (EBSCOhost), and PsycINFO (EBSCOhost) databases from inception to February 10, 2021. Eligible studies were reports developing a disease-related knowledge measurement instrument or assessing one or more of its measurement properties. We assessed the methodological quality of the included studies independently using the “COSMIN Risk of Bias” checklist. We graded the quality of the evidence using a GRADE approach.

      Results

      Twenty-four studies provided information on 14 measurement instruments. All instruments showed sufficient evidence for content validity, three for structural validity, and seven for hypothesis testing for construct validity. Cross-cultural validity and criterion validity were not assessed in any instrument. Only two instruments showed sufficient evidence for the internal consistency of their scores, and two others for their test–retest reliability. Responsiveness was assessed in one instrument, but it was rated as indeterminate.

      Conclusion

      Based on the available evidence, two instruments can be recommended for use, two are unrecommended, and five have the potential to be recommended for use but require further research.

      Keywords

      What is new?

        Key findings

      • There are evidence gaps in the available measurement instruments.
      • Only 2 out of 14 identified measurement instruments are suitable.
      • Five measurement instruments could potentially be recommended for use.

        What this add to what is known?

      • First systematic review on multiple sclerosis patient knowledge measures.

        What is the implication and what should change now?

      • Review findings will help make evidence-based decisions about using these instruments.

      1. Introduction

      Multiple sclerosis (MS) affects an estimated 2.3 million people worldwide, with a prevalence of 50–300 per 100,000 inhabitants [
      • Thompson A.J.
      • Baranzini S.E.
      • Geurts J.
      • Hemmer B.
      • Ciccarelli O.
      Multiple sclerosis.
      ]. It is considered the most common demyelinating disease and is the first cause of non–traumatic neurological disability in young adults [
      • Filippi M.
      • Bar-Or A.
      • Piehl F.
      • Preziosa P.
      • Solari A.
      • Vukusic S.
      • et al.
      Multiple sclerosis.
      ]. In recent decades, the review of MS diagnostic criteria, the emergence of new therapies, and the identification of predictive biomarkers have enabled early detection and treatment of the disease [
      • Thompson A.J.
      • Baranzini S.E.
      • Geurts J.
      • Hemmer B.
      • Ciccarelli O.
      Multiple sclerosis.
      ,
      • Filippi M.
      • Bar-Or A.
      • Piehl F.
      • Preziosa P.
      • Solari A.
      • Vukusic S.
      • et al.
      Multiple sclerosis.
      ,
      • Wingerchuk D.M.
      • Weinshenker B.G.
      Disease modifying therapies for relapsing multiple sclerosis.
      ]. These aspects, combined with increasingly tailored therapeutic decisions, have allowed early treatment, thus reducing relapse rates and slowing down disease progression. Moreover, an active person-centered approach [
      • Bokhour B.G.
      • Fix G.M.
      • Mueller N.M.
      • Barker A.M.
      • Lavela S.L.
      • Hill J.N.
      • et al.
      How can healthcare organizations implement patient-centered care? Examining a large-scale cultural transformation.
      ,
      • Heesen C.
      • Köpke S.
      • Solari A.
      • Geiger F.
      • Kasper J.
      Patient autonomy in multiple sclerosis — possible goals and assessment strategies.
      ] is being promoted at all stages of the disease to minimize its impact, maximize the quality of life, and adopt a wellness philosophy [
      • Thompson A.J.
      • Baranzini S.E.
      • Geurts J.
      • Hemmer B.
      • Ciccarelli O.
      Multiple sclerosis.
      ]. In this paradigm of contemplating preferences, needs and expectations of people with MS, good professional–patient communication, and shared decision-making should prevail [
      • Bokhour B.G.
      • Fix G.M.
      • Mueller N.M.
      • Barker A.M.
      • Lavela S.L.
      • Hill J.N.
      • et al.
      How can healthcare organizations implement patient-centered care? Examining a large-scale cultural transformation.
      ,
      • Heesen C.
      • Köpke S.
      • Solari A.
      • Geiger F.
      • Kasper J.
      Patient autonomy in multiple sclerosis — possible goals and assessment strategies.
      ,
      • Colligan E.
      • Metzler A.
      • Tiryaki E.
      Shared decision-making in multiple sclerosis.
      ]. To reach this optimal point, contributing to, ensuring, and improving patients’ MS-related knowledge should be the first steps on this lifelong path.
      In health education, knowledge is defined as the “factual and interpretive information leading to understanding or usefulness for taking informed action” [
      Communication theory and health behavior change.
      ]. As such, the disease-related knowledge of people with MS can influence disease self-management, coping, and adherence, which consequently affect clinical outcomes [
      • Vermersch P.
      • Shanahan J.
      • Langdon D.
      • Yeandle D.
      • Alexandri N.
      • Schippling S.
      Knowledge is power, but is ignorance bliss? Optimising conversations about disease progression in multiple sclerosis.
      ]. Furthermore, it is a requirement for shared decision-making, a key component of patient-centered healthcare that is critical in chronic diseases such as MS [
      • Heesen C.
      • Köpke S.
      • Solari A.
      • Geiger F.
      • Kasper J.
      Patient autonomy in multiple sclerosis — possible goals and assessment strategies.
      ]. Likewise, patient knowledge is a relevant outcome to measure the effectiveness of strategies for informing, educating, and involving patients [
      • Coulter A.
      • Ellins J.
      Effectiveness of strategies for informing, educating, and involving patients.
      ]. There is a need to use measurement instruments in research and practice with sufficient evidence to evaluate this outcome in a given population and context. However, several studies that evaluate the impact of information provision interventions in disease-related knowledge using measurement instruments fail to report or assess their validity [
      • Köpke S.
      • Solari A.
      • Rahn A.
      • Khan F.
      • Heesen C.
      • Giordano A.
      Information provision for people with multiple sclerosis. Cochrane multiple sclerosis and rare diseases of the CNS group.
      ].
      Systematic reviews of outcome measurement instruments assess their quality and characteristics to determine the most suitable ones for use in clinical practice, health service planning, and research [
      • Prinsen C.A.C.
      • Mokkink L.B.
      • Bouter L.M.
      • Alonso J.
      • Patrick D.L.
      • de Vet H.C.W.
      • et al.
      COSMIN guideline for systematic reviews of patient-reported outcome measures.
      ]. Because we found none with this approach either in the literature or in the prospective records of systematic reviews (PROSPERO database), we conducted a systematic review to summarize the evidence on the measurement properties of available disease-related knowledge measurement instruments of people with MS and identify the most suitable ones.

      2. Materials and methods

      This review has been conducted following the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) initiative [
      • Prinsen C.A.C.
      • Mokkink L.B.
      • Bouter L.M.
      • Alonso J.
      • Patrick D.L.
      • de Vet H.C.W.
      • et al.
      COSMIN guideline for systematic reviews of patient-reported outcome measures.
      ,
      • Mokkink L.B.
      • de Vet H.C.W.
      • Prinsen C.A.C.
      • Patrick D.L.
      • Alonso J.
      • Bouter L.M.
      • et al.
      COSMIN risk of bias checklist for systematic reviews of patient-reported outcome measures.
      ,
      • Terwee C.B.
      • Prinsen C.A.C.
      • Chiarotto A.
      • Westerman M.J.
      • Patrick D.L.
      • Alonso J.
      • et al.
      COSMIN methodology for evaluating the content validity of patient-reported outcome measures: a Delphi study.
      ] and reported in accordance with the Preferred Reporting Items for Systematic reviews and Meta-Analyses 2020 statement [
      • Page M.J.
      • McKenzie J.E.
      • Bossuyt P.M.
      • Boutron I.
      • Hoffmann T.C.
      • Mulrow C.D.
      • et al.
      The PRISMA 2020 statement: an updated guideline for reporting systematic reviews.
      ] and its literature search extension [
      • Rethlefsen M.L.
      • Kirtley S.
      • Waffenschmidt S.
      • Waffenschmidt S.
      • Ayala A.P.
      • Moher D.
      • et al.
      PRISMA-S: an extension to the PRISMA statement for reporting literature searches in systematic reviews.
      ].

      2.1 Eligibility criteria and information sources

      Studies were eligible if they met the following criteria: (1) the instrument aimed to measure disease-related knowledge, (2) no less than 75% of the study population consisted of people with MS, and (3) the aim of the study is to develop an instrument that assessed one or more of its measurement properties or evaluated its interpretability and feasibility. Development studies of the identified instruments were also included, even if they did not involve people with MS, as such studies could provide indirect evidence on the instrument’s content validity. Studies that only used the instrument as an outcome measure (e.g., clinical trials) or for validation of other instruments were excluded, as were conference abstracts. We performed a literature search without language restrictions in the MEDLINE (PubMed), CINAHL (EBSCOhost), PsycINFO (EBSCOhost), and OpenGrey and Grey Literature Report databases from their inception until February 10, 2021. We also screened the reference lists of included reports, complemented the main search with an additional one using only the instrument’s name, and contacted the authors of the included studies to retrieve the maximum possible information about the instruments identified.

      2.2 Search strategy and selection process

      Terms in controlled language and free text were combined. Likewise, we added a highly sensitive filter developed by COSMIN for the MEDLINE (PubMed) search to identify studies on the measurement properties of the instruments [
      • Terwee C.B.
      • Jansma E.P.
      • Riphagen I.I.
      • de Vet H.C.W.
      Development of a methodological PubMed search filter for finding studies on measurement properties of measurement instruments.
      ]. Reproducible searches for all databases are available at https://doi.org/10.5281/zenodo.5166552. We imported the retrieved records into the Rayyan QCRI web application program [
      • Ouzzani M.
      • Hammady H.
      • Fedorowicz Z.
      • Elmagarmid A.
      Rayyan—a web and mobile app for systematic reviews.
      ]. Two reviewers manually removed duplicates after using Rayyan QCRI’s duplicate identification strategy. These reviewers independently screened the titles and abstracts of the records obtained, confronting them with the eligibility criteria. If a record seemed relevant to at least one of the reviewers, they independently reviewed the full-text report. Conflicts over inclusion were discussed, and a third reviewer was consulted in case of disagreement.

      2.3 Data collection process and data items

      We collected the information on the included studies and the identified instruments in the data extraction spreadsheets developed by COSMIN [
      COnsensus-based Standards for the selection of health Measurement INstruments. COSMIN tools. COSMIN.
      ]. The included studies were grouped by instrument to identify the number of studies and instruments separately. A modified instrument was treated as a new instrument [
      • Prinsen C.A.C.
      • Mokkink L.B.
      • Bouter L.M.
      • Alonso J.
      • Patrick D.L.
      • de Vet H.C.W.
      • et al.
      COSMIN guideline for systematic reviews of patient-reported outcome measures.
      ,
      • Mokkink L.B.
      • de Vet H.C.W.
      • Prinsen C.A.C.
      • Patrick D.L.
      • Alonso J.
      • Bouter L.M.
      • et al.
      COSMIN risk of bias checklist for systematic reviews of patient-reported outcome measures.
      ,
      • Terwee C.B.
      • Prinsen C.A.C.
      • Chiarotto A.
      • Westerman M.J.
      • Patrick D.L.
      • Alonso J.
      • et al.
      COSMIN methodology for evaluating the content validity of patient-reported outcome measures: a Delphi study.
      ]. Two reviewers independently extracted the data from included studies and instruments. Disagreements were resolved by discussion between these reviewers; if no agreement could be reached, a third reviewer decided.

      2.4 Study risk of bias assessment

      We assessed the methodological quality of each study using the COSMIN Risk of Bias checklist. We analyzed the following measurement properties: content validity, construct validity (structural validity, hypothesis testing, and cross-cultural validity), criterion validity, reliability (internal consistency, test–retest reliability, and measurement error), and responsiveness. Concerning criterion validity, we agreed, based on the COSMIN guidelines [
      • Prinsen C.A.C.
      • Mokkink L.B.
      • Bouter L.M.
      • Alonso J.
      • Patrick D.L.
      • de Vet H.C.W.
      • et al.
      COSMIN guideline for systematic reviews of patient-reported outcome measures.
      ,
      • Mokkink L.B.
      • de Vet H.C.W.
      • Prinsen C.A.C.
      • Patrick D.L.
      • Alonso J.
      • Bouter L.M.
      • et al.
      COSMIN risk of bias checklist for systematic reviews of patient-reported outcome measures.
      ], that no gold standard exists for identified instruments. Moreover, we did not consider P-values but the direction and magnitude of observed correlations in assessing hypothesis testing for construct validity and responsiveness [
      • Prinsen C.A.C.
      • Mokkink L.B.
      • Bouter L.M.
      • Alonso J.
      • Patrick D.L.
      • de Vet H.C.W.
      • et al.
      COSMIN guideline for systematic reviews of patient-reported outcome measures.
      ,
      • Mokkink L.B.
      • de Vet H.C.W.
      • Prinsen C.A.C.
      • Patrick D.L.
      • Alonso J.
      • Bouter L.M.
      • et al.
      COSMIN risk of bias checklist for systematic reviews of patient-reported outcome measures.
      ,
      • Terwee C.B.
      • Prinsen C.A.C.
      • Chiarotto A.
      • Westerman M.J.
      • Patrick D.L.
      • Alonso J.
      • et al.
      COSMIN methodology for evaluating the content validity of patient-reported outcome measures: a Delphi study.
      ]. The review team agreed that correlations of at least 0.50 between the instrument under study and a comparison instrument measuring the same construct, and correlations of 0.30–0.50 between instruments measuring related but different constructs, would be interpreted as adequate. Interpretability and feasibility are not considered measurement properties but are essential aspects when selecting a measurement instrument, so they were compiled in specific tables. We discussed a priori how ratings should be determined and piloted the ratings with some articles from the review to take their scope into account. Two reviewers rated all of the studies independently, which were then discussed and agreed upon by the review team.

      2.5 Synthesis methods and quality of evidence

      Regarding content validity, the results of each study were rated by two reviewers independently using the 10 criteria for good content validity established by the COSMIN guidelines [
      • Terwee C.B.
      • Prinsen C.A.C.
      • Chiarotto A.
      • Westerman M.J.
      • Patrick D.L.
      • Alonso J.
      • et al.
      COSMIN methodology for evaluating the content validity of patient-reported outcome measures: a Delphi study.
      ]. In addition, the reviewers rated the content of the instruments themselves. Each criterion could be rated as sufficient, insufficient, or indeterminate. Subsequently, the results of all the studies on a specific instrument and the reviewers’ ratings were summarized qualitatively. The review team agreed on an overall rating of sufficient, insufficient, or inconsistent for the content validity of each instrument. For the other measurement properties, the results were rated according to the updated criteria for good measurement properties [
      • Prinsen C.A.C.
      • Mokkink L.B.
      • Bouter L.M.
      • Alonso J.
      • Patrick D.L.
      • de Vet H.C.W.
      • et al.
      COSMIN guideline for systematic reviews of patient-reported outcome measures.
      ]. We subsequently graded the quality of their evidence using a “Grading of Recommendations Assessment, Development and Evaluation (GRADE)” approach modified by COSMIN. This approach uses four factors to determine the quality of the evidence: (1) risk of bias, (2) inconsistency of the results of the studies, (3) inaccuracy, and (4) indirect evidence. We made recommendations on the use of each identified instrument based on the available evidence and its quality grade. We classified them into three categories according to COSMIN guidelines: (A) instruments whose content validity had sufficient evidence and at least a low quality of evidence for a sufficient internal consistency of its scores, (B) instruments with high-quality evidence for an insufficient measurement property, and (C) instruments not classified either as A or B. Instruments classified as A were recommended for use while those classified as B were not. Instruments classified as C had the potential to be recommended, but further studies were needed to assess their quality.

      2.6 Standard protocol approvals, registrations, and patient consents

      Ethical approval and participant consent were not necessary as it is a review based solely on published studies. The review protocol was prospectively registered in the International Prospective Register of Systematic Reviews (PROSPERO CRD42019125417); no changes were made to this protocol. Details of the rationale and design of the review have been previously published [
      • Gonzalez-del-Rio M.
      • Bertran-Noguer C.
      • Ramió-Torrentà L.
      • Zabaleta-del-Olmo E.
      Disease-related knowledge measurement instruments of people affected by multiple sclerosis: protocol for a systematic psychometric review.
      ].

      3. Results

      The literature search and study selection process are detailed in Fig. 1. Twenty-five reports [
      • Rahn A.C.
      • Backhus I.
      • Fuest F.
      • Riemann-Lorenz K.
      • Köpke S.
      • van de Roemer A.
      • et al.
      Comprehension of confidence intervals - development and piloting of patient information materials for people with multiple sclerosis: qualitative study and pilot randomised controlled trial.
      ,
      • Maybury C.P.
      • Brewin C.R.
      Social relationships, knowledge and adjustment to multiple sclerosis.
      ,
      • Dehghani A.
      • Keshavarzi A.
      Development and validation of a multidimensional health literacy questionnaire for multiple sclerosis patients.
      ,
      • Giordano A.
      • Messmer Uccelli M.
      • Pucci E.
      • Martinelli V.
      • Borreani C.
      • Lugaresi A.
      • et al.
      The Multiple Sclerosis Knowledge Questionnaire: a self-administered instrument for recently diagnosed patients.
      ,
      • Bishop M.
      • Frain M.
      Development and initial analysis of multiple sclerosis self-management scale.
      ,
      • Bishop M.
      • Frain M.P.
      • Tschopp M.K.
      Self-management, perceived control, and subjective quality of life in multiple sclerosis: an exploratory study.
      ,
      • Bishop M.
      • Frain M.P.
      • Rumrill P.D.
      • Rymond C.
      The relationship of self-management and disease modifying therapy use to employment status among adults with multiple sclerosis.
      ,
      • Bishop M.
      • Frain M.P.
      The multiple sclerosis self-management scale: revision and psychometric analysis.
      ,
      • Ghahari S.
      • Khoshbin L.S.
      • Forwell S.J.
      The multiple sclerosis self-management scale.
      ,
      • Wilski M.
      • Tasiemski T.
      • Kocur P.
      Demographic, socioeconomic and clinical correlates of self-management in multiple sclerosis.
      ,
      • Wilski M.
      • Tasiemski T.
      Illness perception, treatment beliefs, self-esteem, and self-efficacy as correlates of self-management in multiple sclerosis.
      ,
      • Erbay Ö.
      • Usta Yeşilbalkan Ö.
      • Yüceyar N.
      • Baklan M.
      • Karadakovan A.
      • Tekindal M.A.
      Validity and reliability study of the Turkish version of multiple sclerosis self-management scale.
      ,
      • Saadat S.
      • Kajbaf M.B.
      • Kalantari M.
      • Hosseininezhad M.
      The multiple sclerosis self-management scale–revised (MSSM-R).
      ,
      • Tomczak M.
      • Kleka P.
      • Wilski M.
      Psychometric properties of the polish version of the multiple sclerosis self-management scale – revised.
      ,
      • Bishop M.L.
      • Frain M.P.
      • Li J.
      • Chiu C.Y.
      • McDaniels B.
      • Kim B.J.
      The multiple sclerosis self-management scale-2: evaluation of an updated scale.
      ,
      • Hibbard J.H.
      • Stockard J.
      • Mahoney E.R.
      • Tusler M.
      Development of the patient activation measure (PAM): conceptualizing and measuring activation in patients and consumers: development of the patient activation measure (PAM).
      ,
      • Hibbard J.H.
      • Mahoney E.R.
      • Stockard J.
      • Tusler M.
      Development and testing of a short form of the patient activation measure.
      ,
      • Stepleman L.
      • Rutter M.C.
      • Hibbard J.
      • Johns L.
      • Wright D.
      • Hughes M.
      Validation of the patient activation measure in a multiple sclerosis clinic sample and implications for care.
      ,
      • Goodworth M.C.R.
      • Stepleman L.
      • Hibbard J.
      • Johns L.
      • Wright D.
      • Hughes M.D.
      • et al.
      Variables associated with patient activation in persons with multiple sclerosis.
      ,
      • Heesen C.
      • Kasper J.
      • Segal J.
      • Köpke S.
      • Mühlhauser I.
      Decisional role preferences, risk knowledge and information interests in patients with multiple sclerosis.
      ,
      • Heesen C.
      • Kasper J.
      • Fischer K.
      • Köpke S.
      • Rahn A.
      • Backhus I.
      • et al.
      Risk knowledge in relapsing multiple sclerosis (RIKNO 1.0) - development of an outcome instrument for educational interventions.
      ,
      • Heesen C.
      • Pöttgen J.
      • Rahn A.C.
      • Liethmann K.
      • Kasper J.
      • Vahter L.
      • et al.
      What should a person with relapsing-remitting multiple sclerosis know? – focus group and survey data of a risk knowledge questionnaire (RIKNO 2.0).
      ,
      • Giordano A.
      • Liethmann K.
      • Köpke S.
      • Poettgen J.
      • Rahn A.C.
      • Drulovic J.
      • et al.
      Risk knowledge of people with relapsing-remitting multiple sclerosis – results of an international survey.
      ,
      • Abolfazli R.
      • Elyasi A.
      • Javadi M.R.
      • Gholami K.
      • Torkamandi H.
      • Amir-Shahkarami M.
      • et al.
      Knowledge and attitude assessment of Iranian multiple sclerosis patients receiving interferon beta.
      ,
      • Rath L.
      • Vijiaratnam N.
      • Skibina O.
      Assessing understanding of individual risk and symptoms of progressive multifocal leukoencephalopathy in patients prescribed natalizumab for multiple sclerosis: patient understanding: natalizumab-PML.
      ], involving 24 studies, were included. These studies provided information about the measurement properties of 14 different instruments, whose characteristics are presented in Tables 1 and 2. The reasons for excluding reports that initially seemed to meet eligibility criteria are listed in Supplementary Text A. The results of the quality assessment of each study are shown in Tables 3 and 4. The rating of every study against the criteria for good measurement properties is described in Supplementary Table A. Finally, the summary of findings by measurement instrument is presented in Table 5.
      Figure thumbnail gr1
      Fig. 1PRISMA 2020 flow diagram. Abbreviation: PRISMA, Preferred Reporting Items for Systematic reviews and Meta-Analyses.
      Table 1Characteristics of the identified measurement instruments
      Measurement instrument (reference to the first article)Construct(s)Target populationMode of administrationRecall period(Sub)scale(s) (number of items)Response optionsRange of scores/scoringOriginal languageAvailable translations
      Comprehension of Confidence Intervals Questionnaire (Rahn et al. 2016) [
      • Rahn A.C.
      • Backhus I.
      • Fuest F.
      • Riemann-Lorenz K.
      • Köpke S.
      • van de Roemer A.
      • et al.
      Comprehension of confidence intervals - development and piloting of patient information materials for people with multiple sclerosis: qualitative study and pilot randomised controlled trial.
      ]
      Comprehension of confidence intervalsPeople affected by MSSelf-administrationNow6 itemsMultiple choice with one correct answerTotal score: correct answers summation

      Range of scores: 0–6

      Higher score = higher comprehension
      GermanNIA
      Knowledge of Multiple Sclerosis Scale (Maybury and Brewin 1984) [
      • Maybury C.P.
      • Brewin C.R.
      Social relationships, knowledge and adjustment to multiple sclerosis.
      ]
      Knowledge about MSPeople affected by MSSelf- administration and interview administrationNow14 questionsMultiple choice with multiple correct answersTotal score: summation of the assigned score (0, 1, 2) on each question

      Range of scores: 0–28

      Higher score = higher knowledge
      English (UK)NIA
      Multiple Sclerosis Health Literacy Questionnaire (Dehghani and Keshavarzi 2018) [
      • Dehghani A.
      • Keshavarzi A.
      Development and validation of a multidimensional health literacy questionnaire for multiple sclerosis patients.
      ]
      Health literacyPeople affected by MSSelf- administrationNow22 items divided into 4 subscales: (1) appraisal of health information (5 items), (2) the ability to search health information (5 items), (3) the knowledge of caring for the disease (7 items), and (4) successful practices in health conditions (5 items)Five-point scaleTotal score: summation of the assigned score (1–5) to each item

      Range of scores: 22–110

      Higher score = higher health literacy
      PersianNIA
      Multiple Sclerosis Knowledge Questionnaire (Giordano et al. 2010) [
      • Giordano A.
      • Messmer Uccelli M.
      • Pucci E.
      • Martinelli V.
      • Borreani C.
      • Lugaresi A.
      • et al.
      The Multiple Sclerosis Knowledge Questionnaire: a self-administered instrument for recently diagnosed patients.
      ]
      Knowledge about diseaseRecently diagnosed MS peopleSelf- administrationNow25 items

      Two MSKQ versions (A and B) differ in item order, with version B intended for readministration
      Multiple choice with one correct answerTotal score: correct answers summation

      Range of scores: 0–25

      Higher score = higher knowledge
      ItalianDutch

      English

      German

      Turkish
      Multiple Sclerosis Self-Management scale (Bishop and Frain 2007) [
      • Bishop M.
      • Frain M.
      Development and initial analysis of multiple sclerosis self-management scale.
      ]
      Self-management knowledge and behaviourPeople affected by MSSelf- administrationNow38 items divided into 7 subscales: (1) treatment adherence (7 items); (2) care provider–patient relationship (5 items); (3) emotional health and social support (8 items); (4) health and symptom awareness (4 items); (5) MS knowledge and information (5 items); (6) health maintenance behaviour (5 items); and (7) communication about symptoms (4 items)5-point Likert-type scaleTotal score: summation of the assigned score (1–5) to each item

      Range of scores: 39–195 transformed to a scaled score (0–100) with a formula

      Higher score = higher self-management
      English (US)NIA
      Multiple Sclerosis Self-Management scale-revised (Bishop and Frain 2011) [
      • Bishop M.
      • Frain M.P.
      The multiple sclerosis self-management scale: revision and psychometric analysis.
      ]
      Self-management knowledge and behaviourPeople affected by MSSelf- administrationNow24 items divided into 5 subscales: (1) healthcare provider relationship (6 items); (2) treatment adherence/barriers (7 items); (3) social/family support (3 items); (4) MS knowledge and information (4 items); and (5) health maintenance behaviour (4 items)5-point Likert-type scaleTotal score: summation of the assigned score (1–5) to each item, three of them with a reverse score

      Range of scores: 24–120 transformed to a scaled score (0–100) with a formula

      Higher score = higher self-management
      English (US)English (Canada)

      Persian

      Polish

      Turkish
      Multiple Sclerosis Self-Management scale-2 (Bishop et al. 2019) [
      • Bishop M.L.
      • Frain M.P.
      • Li J.
      • Chiu C.Y.
      • McDaniels B.
      • Kim B.J.
      The multiple sclerosis self-management scale-2: evaluation of an updated scale.
      ]
      Self-management knowledge and behaviourPeople affected by MSSelf- administrationNow29 items divided into 7 subscales: (1) healthcare provider relationship/communication (8 items); (2) health promotion engagement (5 items); (3) treatment adherence (3 items); (4) social/family support (3 items); (5) MS knowledge and information (3 items), (6) health maintenance behaviour/prevention (4 items); and (7) treatment adherence barriers (3 items)5-point Likert-type scaleTotal score: summation of the assigned score (1–5) to each item

      Range of scores: 29–145 transformed to a scaled score (0–100) with a formula

      Higher scores = higher self-management
      English (US)NIA
      Patient Activation Measure (Hibbard et al. 2004) [
      • Hibbard J.H.
      • Stockard J.
      • Mahoney E.R.
      • Tusler M.
      Development of the patient activation measure (PAM): conceptualizing and measuring activation in patients and consumers: development of the patient activation measure (PAM).
      ]
      Patient activation (knowledge, skills, and confidence in self-management on health or chronic condition)People affected by chronic conditionsSelf- administration and interview administrationNowUnidimensional scale (22 items/statements)Five possible responses: strongly agree, agree, disagree, strongly disagree, not applicableTotal score: summation of the assigned score (1–4) to each item, one of them with a reverse score

      “Not Applicable” answers: divide the score by the number of items completed and multiply by 22. Scores are not calculated for respondents who gave eight or more “Not Applicable” answers

      Higher scores = higher activation
      English (US)Portuguese (Brazil)
      Patient Activation Measure short form (Hibbard et al. 2005) [
      • Hibbard J.H.
      • Mahoney E.R.
      • Stockard J.
      • Tusler M.
      Development and testing of a short form of the patient activation measure.
      ]
      Patient activation (knowledge, skills, and confidence in self-management of one’s health or chronic condition)People affected by chronic conditionsSelf- administration and interview administrationNowUnidimensional scale (13 items/statements)Five possible responses: strongly agree, agree, disagree, strongly disagree, not applicableTotal score: [raw score]/[# items answered excepting nonapplicable items] × 13; can be transformed to a scale with a theoretical range 0–100

      Levels of patient activation:
      • Level 1: score under 47
      • Level 2: score between 47.1 and 55.2
      • Level 3: score between 55.2 and 67.0
      • Level 4: score above 67.1
      Higher scores = higher activation
      English (US)Portuguese

      German

      Danish

      Italian

      Hebrew

      French

      Korean

      Spanish (Spain)

      Norwegian

      Swedish
      Risk Knowledge questionnaire (Heesen et al. 2004) [
      • Heesen C.
      • Kasper J.
      • Segal J.
      • Köpke S.
      • Mühlhauser I.
      Decisional role preferences, risk knowledge and information interests in patients with multiple sclerosis.
      ]
      MS risk knowledgePeople affected by MSSelf- administrationNow19 itemsMultiple choice with one correct answerTotal score: correct answers summation. Missing answers are considered as wrong

      Range of scores: 0–19

      Higher score = higher knowledge
      GermanNIA
      Risk Knowledge 1.0 questionnaire (Heesen et al. 2015) [
      • Heesen C.
      • Kasper J.
      • Fischer K.
      • Köpke S.
      • Rahn A.
      • Backhus I.
      • et al.
      Risk knowledge in relapsing multiple sclerosis (RIKNO 1.0) - development of an outcome instrument for educational interventions.
      ]
      MS risk knowledgePeople affected by early RRMSSelf- administrationNow19 items divided into 5 categories: (1) general MS issues (4 items), (2) diagnosis (4 items), (3) prognosis (4 items), (4) treatment (5 items), and (5) evidence-based medicine (2 items)Multiple choice with one correct answerTotal score: correct answers summation

      Range of scores: 0–19

      Higher score = higher knowledge
      GermanNIA
      Risk Knowledge 2.0 questionnaire (Heesen et al. 2017) [
      • Heesen C.
      • Pöttgen J.
      • Rahn A.C.
      • Liethmann K.
      • Kasper J.
      • Vahter L.
      • et al.
      What should a person with relapsing-remitting multiple sclerosis know? – focus group and survey data of a risk knowledge questionnaire (RIKNO 2.0).
      ]
      MS risk knowledgePeople affected by MSSelf- administrationNow19 itemsMultiple choice with one correct answerTotal score: correct answers summation

      Range of scores: 0–21

      Higher score = higher knowledge
      GermanDutch

      English (UK)

      Estonian

      Flemish

      French

      Italian

      Serbian

      Spanish (Spain)

      Turkish
      Unnamed 1 (Abolfazli et al. 2014) [
      • Abolfazli R.
      • Elyasi A.
      • Javadi M.R.
      • Gholami K.
      • Torkamandi H.
      • Amir-Shahkarami M.
      • et al.
      Knowledge and attitude assessment of Iranian multiple sclerosis patients receiving interferon beta.
      ]
      Perspectives and knowledge regarding treatmentPeople affected by MS receiving interferon betaSelf- administrationNow25 questions: 12 items related to knowledge and 13 to attitudeKnowledge questions: multiple choice with one correct answer

      Attitude questions: 5-point Likert scale
      Total score: correct answers summation

      Range of knowledge scores: 0–12

      Range of attitude scores: 13–65

      Higher scores = higher knowledge and better attitudes
      PersianNIA
      Unnamed 2 (Rath et al. 2017) [
      • Rath L.
      • Vijiaratnam N.
      • Skibina O.
      Assessing understanding of individual risk and symptoms of progressive multifocal leukoencephalopathy in patients prescribed natalizumab for multiple sclerosis: patient understanding: natalizumab-PML.
      ]
      Knowledge and understanding of risk and symptoms of PMLPeople affected by MS in treatment with natalizumabSelf- administrationNow18 questions divided into 6 focus areas: (1) basic PML knowledge; (2) preinfusion questionnaire compliance; (3) wallet alert card compliance; (4) co-ownership of surveillance tests; (5) involvement desired in risk management; and (6) knowledge of other factors affecting riskMultiple choice with a correct answerTotal score: correct answers summation

      Range of scores: 0–18

      Higher score = higher knowledge
      English (Australia)NIA
      Abbreviations: MS, multiple sclerosis; MSKQ, Multiple Sclerosis Knowledge Questionnaire; NIA, no information available; PML, progressive multifocal leukoencephalopathy; RRMS, relapsing-remitting multiple sclerosis.
      Table 2Characteristics of the included studies
      Measurement instrumentReferencePopulationDisease/Condition characteristicsInstrument administration
      NAge, mean (SD, range)FemaleEducation levelSocioeconomic characteristicsMS duration (yr), mean (SD)Disease severitySettingCountryLanguageResponse rate
      Comprehension of Confidence Intervals QuestionnaireRahn et al. 2016 [
      • Rahn A.C.
      • Backhus I.
      • Fuest F.
      • Riemann-Lorenz K.
      • Köpke S.
      • van de Roemer A.
      • et al.
      Comprehension of confidence intervals - development and piloting of patient information materials for people with multiple sclerosis: qualitative study and pilot randomised controlled trial.
      ]
      64IG: 47.3 yr

      CG: 43.8 yr
      64.1%Secondary school: 48.4%

      Academic degree: 51.6%
      NIAIG: 9.1 yr

      CG: 9.5 yr
      RRMS: 65.6%

      SPMS: 20.3%

      PPMS: 3.1%

      CIS: 3.1%
      MS day hospital

      Evaluative
      GermanyGerman55.7%
      Knowledge of Multiple Sclerosis ScaleMaybury and Brewin 1984 [
      • Maybury C.P.
      • Brewin C.R.
      Social relationships, knowledge and adjustment to multiple sclerosis.
      ]
      3642.0 yr66.6%NIAEmployed: 22.2%3.7 yrAcute phase or progressing: 19.4%

      Stable phase: 80.6%
      Hospital and Welfare Officer

      Evaluative
      UKEnglish100.0%
      MSHLQDehghani and Keshavarzi 2018 [
      • Dehghani A.
      • Keshavarzi A.
      Development and validation of a multidimensional health literacy questionnaire for multiple sclerosis patients.
      ]
      21031.9 (7.1) yr62.8%Under diploma: 10.9%

      Diploma: 16.7%

      Upper diploma: 72.4%
      NI7.4 (7.3) yrRRMS: 91.9%

      SPMS: 0.0%

      PPMS: 8.1%
      MS Society

      Evaluative
      IranPersianNIA
      MSKQGiordano et al. 2010 [
      • Giordano A.
      • Messmer Uccelli M.
      • Pucci E.
      • Martinelli V.
      • Borreani C.
      • Lugaresi A.
      • et al.
      The Multiple Sclerosis Knowledge Questionnaire: a self-administered instrument for recently diagnosed patients.
      ]
      10235.2 (9.7) yr67.6%Primary: 28.4%

      Secondary: 54.9%

      College: 16.7%
      Employed: 73.5%

      Homemaker: 11.8%

      Student: 10.8%

      Unemployed: 3.9%
      NIARRMS: 91.2%

      PPMS and SPMS: 8.8%
      Two MS centres

      Evaluative

      Research
      ItalyItalian97.0%
      MSSM scaleBishop and Frain 2007 [
      • Bishop M.
      • Frain M.
      Development and initial analysis of multiple sclerosis self-management scale.
      ]
      26645.7 (11.7) yr84.6%Below high school: 1.1%

      High school: 18.5%

      College/technical school: 38.5%

      College graduate: 26.4%

      Master’s degree/higher: 15.5%
      Full time: 40.5%

      Part-time: 5.6%

      Unemployed: 5.6%

      Retired: 9.1%

      Homemakers: 7.1%

      Student: 3.2%
      7.1 (9.3) yrRRMS: 59.2%

      PPMS: 23.5%

      Other: 17.3%
      A regional chapter of the National MS Society

      Evaluative
      USAEnglish53.0%
      Bishop et al. 2008 [
      • Bishop M.
      • Frain M.P.
      • Tschopp M.K.
      Self-management, perceived control, and subjective quality of life in multiple sclerosis: an exploratory study.
      ]
      15745.6 (11.3) yr82.0%NIAFull time: 38.9%

      Part-time: 5.7%

      Unemployed: 55.4%
      7.8 (7.7) yrImmunotherapy: 79.0%Two regional chapters of the National MS Society

      Evaluative
      USAEnglish39.8%
      Bishop et al. 2009 [
      • Bishop M.
      • Frain M.P.
      • Rumrill P.D.
      • Rymond C.
      The relationship of self-management and disease modifying therapy use to employment status among adults with multiple sclerosis.
      ]
      17543.0 (9.0) yr81.0%High school: 18.5%

      College/technical school: 31.2%

      College graduate: 31.2%

      Master’s degree/higher: 11%
      Full-time: 41.6%

      Part-time: 13.9%

      Unemployed: 44.5%
      4.2 (3.8) yrImmunotherapy: 80.0%Three regional chapters of the National MS Society

      Evaluative
      USAEnglish28.0%
      MSSM scale-revisedBishop and Frain 2011 [
      • Bishop M.
      • Frain M.P.
      The multiple sclerosis self-management scale: revision and psychometric analysis.
      ]
      19743.7 (10.4, 21–75) yr82.7%Under high school: 4.1%

      High school diploma: 17.4%

      College/technical school: 36.4%

      College graduates: 30.3%

      Master’s degree/higher: 11.8%
      Full time: 36.9%

      Part-time: 11.8%

      Students: 3.1%

      Homemakers: 4.1%

      Retired: 7.2%

      Permanent disability: 30.8%

      Unemployed: 6.1%
      3.8 (3.1) yrRRM: 79.9%

      SPMS: 4.1%

      PPMS: 5.2%

      Other: 1.5%

      Immunotherapy: 79.0%
      Three chapters of the National Multiple Sclerosis Society

      Evaluative
      USAEnglish36.0%
      Ghahari et al. 2014 [
      • Ghahari S.
      • Khoshbin L.S.
      • Forwell S.J.
      The multiple sclerosis self-management scale.
      ]
      3149.4 (10.7) yr80.6%High school or less: 12.9%

      Diploma/certificate: 19.4%

      College: 51.6%

      Master’s degree/higher: 16.1%
      Full time: 25.8%

      Part-time: 12.9%

      Retired/homemaker: 22.6%

      Disability pension: 45.2%

      Student: 6.5%
      11.8 (8.0) yrRRMS: 61.3%

      PPMS: 9.7%

      SPMS:16.1%

      PRMS: 3.2%
      MS clinics

      Evaluative
      CanadaEnglish87.1%
      Wilski et al. 2015 [
      • Wilski M.
      • Tasiemski T.
      • Kocur P.
      Demographic, socioeconomic and clinical correlates of self-management in multiple sclerosis.
      ]
      28348.2 (11.8) yr65.4%Primary: 1.1%

      Vocational: 13.1%

      Secondary: 36.7%

      Higher: 49.1%
      Employed: 33.2%

      Unemployed: 4.6%

      Disability pension: 47.7%

      Retired: 14.5%
      13.5 (9.6) yrRRMS: 33.2%

      PPMS: 25.1%

      SPMS: 22.6%

      PRMS: 8.8%
      MS Rehabilitation Centre

      Evaluative
      PolandPolish90.4%
      Wilski and Tasiemski 2016 [
      • Wilski M.
      • Tasiemski T.
      Illness perception, treatment beliefs, self-esteem, and self-efficacy as correlates of self-management in multiple sclerosis.
      ]
      21747.0 (10.9) yr66.2%Primary/vocational: 12.4%

      Secondary: 37.1%

      Higher: 50.5%
      Monthly income of one family member
      • 3.3%: <125€
      • 30.0%: 125–250€
      • 23.3%: 250–375€
      • 16.2%: 375–500€
      • 27.2%: >500€
      12.0 (8.0) yrRRMS: 33.8%

      PPMS: 24.3%

      SPMS: 23.8%

      PRMS: 6.7%
      MS rehabilitation clinic

      Evaluative
      PolandPolish97.0%
      Erbay et al. 2020 [
      • Erbay Ö.
      • Usta Yeşilbalkan Ö.
      • Yüceyar N.
      • Baklan M.
      • Karadakovan A.
      • Tekindal M.A.
      Validity and reliability study of the Turkish version of multiple sclerosis self-management scale.
      ]
      24042.1 (10.8) yr70.4%Primary: 35.4%

      High: 30.8%

      University: 32.5%

      Postgraduate: 1.3%
      NIADiagnosed from more than 10 yr, 88.3%RRMS: 94.6%

      SPMS: 5.4%
      Outpatient clinic

      Evaluative
      TurkeyTurkishNIA
      Saadat et al. 2020 [
      • Saadat S.
      • Kajbaf M.B.
      • Kalantari M.
      • Hosseininezhad M.
      The multiple sclerosis self-management scale–revised (MSSM-R).
      ]
      22035.1 (7.4) yr69.1%NIANIA7.4 (4.4) yrNIACommunity-based

      Evaluative
      IranPersian88.0%
      Tomczak et al. 2020 [
      • Tomczak M.
      • Kleka P.
      • Wilski M.
      Psychometric properties of the polish version of the multiple sclerosis self-management scale – revised.
      ]
      66347.1 (11.8, 18–82) yr66.0%Primary/Vocational: 19.5%

      Secondary: 41.6%

      Higher: 38.9%
      Employed: 36.2%

      Unemployed: 5.1%

      Disability pension: 47.9%

      Retiring: 10.8%
      12.3 (9.2) yrRRMS: 37.6%

      PPMS: 24.0%

      SPMS: 20.2%

      PRMS: 8.7%
      Rehabilitation centres and the Polish Society of MS

      Evaluative
      PolandPolishNIA
      MSSM-2 scaleBishop et al. 2019 [
      • Bishop M.L.
      • Frain M.P.
      • Li J.
      • Chiu C.Y.
      • McDaniels B.
      • Kim B.J.
      The multiple sclerosis self-management scale-2: evaluation of an updated scale.
      ]
      2,39358.0 (11.3, 19–96) yr82.9%Under high school: 1.2%

      High school: 20.6%

      College/technical school: 28.3%

      College: 25.0%

      Master’s degree/higher: 24.3%
      Full time: 18.8%

      Part-time: 8.4%

      Students: 0.4%

      Homemakers: 4.3%

      Retired: 18.6%

      Disability pension: 3.9%
      NIANIANational survey

      Evaluative
      USAEnglish34.8%
      PAMHibbard et al. 2004 [
      • Hibbard J.H.
      • Stockard J.
      • Mahoney E.R.
      • Tusler M.
      Development of the patient activation measure (PAM): conceptualizing and measuring activation in patients and consumers: development of the patient activation measure (PAM).
      ]
      1,51545–54 yr: 38.0%

      55–64 yr: 28.0%

      65–74 yr: 20.0%

      75–84 yr: 13.0%

      85 or older: 2.0%
      63.0%High school or less: 43.0%

      College or trade school: 26.0%

      College graduate/higher: 31.0%
      Annual household income:
      • Less than $25,000: 32.0%
      • $25,000–$34,999: 12.0%
      • $35,000–$49,999: 17.0%
      • $50,000–$74.999: 17.0%
      • $75,000 or more: 21.0%
      NIAChronic condition:
      • None: 21.0%
      • Heart problem: 13.0%
      • Arthritis: 38.0%
      • Chronic pain: 25.0%
      • Depression: 15.0%
      • Diabetes: 11.0%
      • Hypertension: 34.0%
      • Lung disease: 12.0%
      • Cancer: 5.0%
      • Dyslipidemia: 30.0%
      National Telephone Survey

      Evaluative
      USAEnglish48.0%
      PAM short formHibbard et al. 2005 [
      • Hibbard J.H.
      • Mahoney E.R.
      • Stockard J.
      • Tusler M.
      Development and testing of a short form of the patient activation measure.
      ]
      1,51545–54 yr: 38.0%

      55–64 yr: 28.0%

      65–74 yr: 20.0%

      75–84 yr: 13.0%

      85 or older: 2.0%
      63.0%High school or less: 43.0%

      College or trade school: 26.0%

      College graduate/higher: 31.0%
      Annual household income
      • Less than $25,000: 32.0%
      • $25,000–$34,999: 12.0%
      • $35,000–$49,999: 17.0%
      • $50,000–$74.999: 17.0%
      • $75,000 or more: 21.0%
      NIAChronic condition:
      • None: 21.0%
      • Heart problem: 13.0%
      • Arthritis: 38.0%
      • Chronic pain: 25.0%
      • Depression: 15.0%
      • Diabetes: 11.0%
      • Hypertension: 34.0%
      • Lung disease: 12.0%
      • Cancer: 5.0%
      • Dyslipidemia: 30.0%
      National Telephone Survey

      Evaluative
      USAEnglish48.0%
      Stepleman et al. 2010 [
      • Stepleman L.
      • Rutter M.C.
      • Hibbard J.
      • Johns L.
      • Wright D.
      • Hughes M.
      Validation of the patient activation measure in a multiple sclerosis clinic sample and implications for care.
      ]/Goodworth et al. 2016 [
      • Goodworth M.C.R.
      • Stepleman L.
      • Hibbard J.
      • Johns L.
      • Wright D.
      • Hughes M.D.
      • et al.
      Variables associated with patient activation in persons with multiple sclerosis.
      ]
      19946.24 (10.8) yr82.0%High school: 22.0%

      College education: 24.0%

      Associate’s degree: 16.0%

      Bachelor’s degree: 16.0%
      Full time: 30.3%

      Part-time: 7.2%

      Unemployed: 55.9%

      Retired: 5.1%

      Disability pension: 1.5%
      8.3 (6.8) yrRRMS: 68.6%

      PPMS: 4.2%

      SPMS: 7.9%

      Unsure: 19.4%
      MS center

      Evaluative
      USAEnglishNIA
      RIKNO questionnaireHeesen et al. 2004 [
      • Heesen C.
      • Kasper J.
      • Segal J.
      • Köpke S.
      • Mühlhauser I.
      Decisional role preferences, risk knowledge and information interests in patients with multiple sclerosis.
      ]
      16944 (11) yr62.7%Higher education: 40.8%NIA7.7 (6.9) yrPPMS: 50.0%

      RRMS: 50.0%

      Immunotherapy: 60.9%

      Early MS: 9.8%
      MS outpatient clinic

      Evaluative
      GermanyGerman79.0%
      RIKNO 1.0 questionnaireHeesen et al. 2015 [
      • Heesen C.
      • Kasper J.
      • Fischer K.
      • Köpke S.
      • Rahn A.
      • Backhus I.
      • et al.
      Risk knowledge in relapsing multiple sclerosis (RIKNO 1.0) - development of an outcome instrument for educational interventions.
      ]
      19236.6 (18-70) yr74.0%University degree: 23.0%

      Secondary school: 52.0%

      Primary school: 25.0%
      NIA1.3 (0-2) yrRRMS: 31.0%

      SPMS: 34.0%

      PPMS: 4.0%

      Immunotherapy: 45.0%

      Early MS: 28.0%
      MS Day Hospital

      Evaluative
      GermanyGerman65.0%
      RIKNO 2.0 questionnaire

      Heesen et al. 2017 [
      • Heesen C.
      • Pöttgen J.
      • Rahn A.C.
      • Liethmann K.
      • Kasper J.
      • Vahter L.
      • et al.
      What should a person with relapsing-remitting multiple sclerosis know? – focus group and survey data of a risk knowledge questionnaire (RIKNO 2.0).
      ]
      70839.8 (10.2) yr26.3%High ≥12 yr: 52%

      Medium (10–11 yr): 35%

      Low (≤9 yr): 13%
      NIA7.1 (6.7) yrDisability
      • Mild: 39.0%
      • Visible; 20.0%
      • Walking aids: 19.0%
      • Wheelchair: 6.0%
      Early MS: 6.0%

      RRMS: 68.0%

      SPMS: 11.0%

      PPMS: 6.0%

      Immunotherapy: 65.0%
      MS Outpatient clinics

      Evaluative
      GermanyGerman62.1%
      Giordano et al. 2018 [
      • Giordano A.
      • Liethmann K.
      • Köpke S.
      • Poettgen J.
      • Rahn A.C.
      • Drulovic J.
      • et al.
      Risk knowledge of people with relapsing-remitting multiple sclerosis – results of an international survey.
      ]
      98638.6 (18–67) yr77.0%NIANIA7.8 (0–37) yrRRMS: 95.0%

      Immunotherapy: 77.0%
      Online survey

      Evaluative
      Germany

      Italy

      The Netherlands

      Serbia

      Spain

      Turkey
      Dutch

      German

      Italian

      Serbian

      Spanish (Spain)

      Turkish
      51.7%
      Unnamed 1Abolfazli et al. 2014 [
      • Abolfazli R.
      • Elyasi A.
      • Javadi M.R.
      • Gholami K.
      • Torkamandi H.
      • Amir-Shahkarami M.
      • et al.
      Knowledge and attitude assessment of Iranian multiple sclerosis patients receiving interferon beta.
      ]
      42534.3 (8.4) yr70.7%High school: 12.2%

      College: 42.4%

      Postgraduate: 8.2%
      NIANIAMean (SD) treatment with interferon beta: 37.2 (27.3) moEvaluativeIranPersian85.0%
      Unnamed 2Rath et al. 2017 [
      • Rath L.
      • Vijiaratnam N.
      • Skibina O.
      Assessing understanding of individual risk and symptoms of progressive multifocal leukoencephalopathy in patients prescribed natalizumab for multiple sclerosis: patient understanding: natalizumab-PML.
      ]
      3720–29 yr: 10.0%

      30–39 yr: 26.0%

      40–49 yr: 37.0%

      50–59 yr: 21.0%

      ≥60 yr: 6.0%
      67.0%NIANIANIATreatment with natalizumab: 48.0%MS-specific clinic in a major tertiary hospital

      Evaluative
      AustraliaEnglish77.1%
      Abbreviations: SD, standard deviation; CG, control group; CIS, clinically isolated syndrome; IG, intervention group; MS, multiple sclerosis; MSHLQ, Multiple Sclerosis Health Literacy Questionnaire; MSKQ, Multiple Sclerosis Knowledge Questionnaire; MSSM, Multiple Sclerosis Self-Management; NIA, no information available; PAM, Patient Activation Measure; PPMS, primary progressive multiple sclerosis; PRMS, progressive-relapsing multiple sclerosis; RRMS, relapsing-remitting multiple sclerosis; RIKNO, Risk Knowledge; SPMS secondary progressive multiple sclerosis.
      Table 3Quality of the measurement instrument development
      Measurement instrumentsDesignCI study
      Empty cells indicate that CI study (or part of it) was not performed.
      Total instrument development
      General design requirementsConcept elicitation
      When the instrument was not developed in a sample representing the target population, the concept elicitation was not rated further.
      Total designGeneral design requirementsComprehensibilityComprehensivenessTotal CI study
      Clear constructClear origin of constructClear target population for which the instrument was developedClear context of useInstrument developed in sample representing the target populationCI study performed in sample representing the target population
      Comprehension of Confidence Intervals Questionnaire [
      • Rahn A.C.
      • Backhus I.
      • Fuest F.
      • Riemann-Lorenz K.
      • Köpke S.
      • van de Roemer A.
      • et al.
      Comprehension of confidence intervals - development and piloting of patient information materials for people with multiple sclerosis: qualitative study and pilot randomised controlled trial.
      ]
      VVVVIIVDDDI
      Knowledge of Multiple Sclerosis Scale [
      • Maybury C.P.
      • Brewin C.R.
      Social relationships, knowledge and adjustment to multiple sclerosis.
      ]
      VDVVIIII
      MSHLQ [
      • Dehghani A.
      • Keshavarzi A.
      Development and validation of a multidimensional health literacy questionnaire for multiple sclerosis patients.
      ]
      VVVVVDDDDDDD
      MSKQ [
      • Giordano A.
      • Messmer Uccelli M.
      • Pucci E.
      • Martinelli V.
      • Borreani C.
      • Lugaresi A.
      • et al.
      The Multiple Sclerosis Knowledge Questionnaire: a self-administered instrument for recently diagnosed patients.
      ]
      VVVVVAAVDDDD
      MSSM scale [
      • Bishop M.
      • Frain M.
      Development and initial analysis of multiple sclerosis self-management scale.
      ]
      VVVVIIADDDI
      MSSM scale-revised [
      • Bishop M.
      • Frain M.P.
      The multiple sclerosis self-management scale: revision and psychometric analysis.
      ]
      VVVVIIII
      MSSM-2 scale [
      • Bishop M.L.
      • Frain M.P.
      • Li J.
      • Chiu C.Y.
      • McDaniels B.
      • Kim B.J.
      The multiple sclerosis self-management scale-2: evaluation of an updated scale.
      ]
      VVVVIIII
      PAM [
      • Hibbard J.H.
      • Stockard J.
      • Mahoney E.R.
      • Tusler M.
      Development of the patient activation measure (PAM): conceptualizing and measuring activation in patients and consumers: development of the patient activation measure (PAM).
      ]
      VVVVVDDAIDII
      PAM short form [
      • Hibbard J.H.
      • Mahoney E.R.
      • Stockard J.
      • Tusler M.
      Development and testing of a short form of the patient activation measure.
      ]
      VVVVVDDAIDII
      RIKNO questionnaire [
      • Heesen C.
      • Kasper J.
      • Segal J.
      • Köpke S.
      • Mühlhauser I.
      Decisional role preferences, risk knowledge and information interests in patients with multiple sclerosis.
      ]
      VVVVDDDADDDD
      RIKNO 1.0 questionnaire [
      • Heesen C.
      • Kasper J.
      • Fischer K.
      • Köpke S.
      • Rahn A.
      • Backhus I.
      • et al.
      Risk knowledge in relapsing multiple sclerosis (RIKNO 1.0) - development of an outcome instrument for educational interventions.
      ]
      VVVVVDDDDDDD
      RIKNO 2.0 questionnaire [
      • Heesen C.
      • Pöttgen J.
      • Rahn A.C.
      • Liethmann K.
      • Kasper J.
      • Vahter L.
      • et al.
      What should a person with relapsing-remitting multiple sclerosis know? – focus group and survey data of a risk knowledge questionnaire (RIKNO 2.0).
      ]
      VVVVVDDVDDDD
      Unnamed 1 [
      • Abolfazli R.
      • Elyasi A.
      • Javadi M.R.
      • Gholami K.
      • Torkamandi H.
      • Amir-Shahkarami M.
      • et al.
      Knowledge and attitude assessment of Iranian multiple sclerosis patients receiving interferon beta.
      ]
      VVVVIIII
      Unnamed 2 [
      • Rath L.
      • Vijiaratnam N.
      • Skibina O.
      Assessing understanding of individual risk and symptoms of progressive multifocal leukoencephalopathy in patients prescribed natalizumab for multiple sclerosis: patient understanding: natalizumab-PML.
      ]
      VVVVIIDDDDI
      Abbreviations: V, very good; A, adequate; D, doubtful; I, inadequate; CI, cognitive interview; MSHLQ, Multiple Sclerosis Health Literacy Questionnaire; MSKQ, Multiple Sclerosis Knowledge Questionnaire; MSSM, Multiple Sclerosis Self-Management; PAM, Patient Activation Measure; RIKNO, Risk Knowledge.
      a Empty cells indicate that CI study (or part of it) was not performed.
      b When the instrument was not developed in a sample representing the target population, the concept elicitation was not rated further.
      Table 4Quality of studies on measurement properties
      Instruments/studiesContent validityStructural validityInternal consistencyCross-cultural validityReliabilityMeasurement errorCriterion validityConstruct validityResponsiveness
      Asking patientsAsking experts
      RelevanceComprehensivenessComprehensibilityRelevanceComprehensivenessConvergent validityKnown groups validityComparison with gold standardComparison with other instrumentsComparison between subgroupsComparison before and after intervention
      Comprehension of Confidence Intervals Questionnaire
       Rahn et al. 2016 [
      • Rahn A.C.
      • Backhus I.
      • Fuest F.
      • Riemann-Lorenz K.
      • Köpke S.
      • van de Roemer A.
      • et al.
      Comprehension of confidence intervals - development and piloting of patient information materials for people with multiple sclerosis: qualitative study and pilot randomised controlled trial.
      ]
      VD
      Knowledge of Multiple Sclerosis Scale
       Maybury and Brewin 1984 [
      • Maybury C.P.
      • Brewin C.R.
      Social relationships, knowledge and adjustment to multiple sclerosis.
      ]
      A
      MSHLQ
       Dehghani and Keshavarzi 2018 [
      • Dehghani A.
      • Keshavarzi A.
      Development and validation of a multidimensional health literacy questionnaire for multiple sclerosis patients.
      ]
      AVDD
      MSKQ
       Giordano et al. 2010 [
      • Giordano A.
      • Messmer Uccelli M.
      • Pucci E.
      • Martinelli V.
      • Borreani C.
      • Lugaresi A.
      • et al.
      The Multiple Sclerosis Knowledge Questionnaire: a self-administered instrument for recently diagnosed patients.
      ]
      DD
      MSSM scale
       Bishop and Frain 2007 [
      • Bishop M.
      • Frain M.
      Development and initial analysis of multiple sclerosis self-management scale.
      ]
      AIV
       Bishop et al. 2008 [
      • Bishop M.
      • Frain M.P.
      • Tschopp M.K.
      Self-management, perceived control, and subjective quality of life in multiple sclerosis: an exploratory study.
      ]
      IV
       Bishop et al. 2009 [
      • Bishop M.
      • Frain M.P.
      • Rumrill P.D.
      • Rymond C.
      The relationship of self-management and disease modifying therapy use to employment status among adults with multiple sclerosis.
      ]
      ID
      MSSM scale-revised
       Bishop and Frain 2011 [
      • Bishop M.
      • Frain M.P.
      The multiple sclerosis self-management scale: revision and psychometric analysis.
      ]
      AVA
       Ghahari et al. 2014 [
      • Ghahari S.
      • Khoshbin L.S.
      • Forwell S.J.
      The multiple sclerosis self-management scale.
      ]
      DDDAA
       Wilski et al. 2015 [
      • Wilski M.
      • Tasiemski T.
      • Kocur P.
      Demographic, socioeconomic and clinical correlates of self-management in multiple sclerosis.
      ]
      I
       Wilski and Tasiemski 2016 [
      • Wilski M.
      • Tasiemski T.
      Illness perception, treatment beliefs, self-esteem, and self-efficacy as correlates of self-management in multiple sclerosis.
      ]
      I
       Erbay et al. 2020 [
      • Erbay Ö.
      • Usta Yeşilbalkan Ö.
      • Yüceyar N.
      • Baklan M.
      • Karadakovan A.
      • Tekindal M.A.
      Validity and reliability study of the Turkish version of multiple sclerosis self-management scale.
      ]
      IAVI
       Saadat et al. 2020 [
      • Saadat S.
      • Kajbaf M.B.
      • Kalantari M.
      • Hosseininezhad M.
      The multiple sclerosis self-management scale–revised (MSSM-R).
      ]
      IVVD
       Tomczak et al. 2020 [
      • Tomczak M.
      • Kleka P.
      • Wilski M.
      Psychometric properties of the polish version of the multiple sclerosis self-management scale – revised.
      ]
      DVVV
      MSSM-2 scale
       Bishop et al. 2019 [
      • Bishop M.L.
      • Frain M.P.
      • Li J.
      • Chiu C.Y.
      • McDaniels B.
      • Kim B.J.
      The multiple sclerosis self-management scale-2: evaluation of an updated scale.
      ]
      VVV
      PAM
       Hibbard et al. 2004 [
      • Hibbard J.H.
      • Stockard J.
      • Mahoney E.R.
      • Tusler M.
      Development of the patient activation measure (PAM): conceptualizing and measuring activation in patients and consumers: development of the patient activation measure (PAM).
      ]
      VVAID
      PAM short form
       Hibbard et al. 2005 [
      • Hibbard J.H.
      • Mahoney E.R.
      • Stockard J.
      • Tusler M.
      Development and testing of a short form of the patient activation measure.
      ]
      VVD
       Stepleman et al. 2010 [
      • Stepleman L.
      • Rutter M.C.
      • Hibbard J.
      • Johns L.
      • Wright D.
      • Hughes M.
      Validation of the patient activation measure in a multiple sclerosis clinic sample and implications for care.
      ]/Goodworth et al. 2016 [
      • Goodworth M.C.R.
      • Stepleman L.
      • Hibbard J.
      • Johns L.
      • Wright D.
      • Hughes M.D.
      • et al.
      Variables associated with patient activation in persons with multiple sclerosis.
      ]
      AVVD
      RIKNO questionnaire
       Heesen et al. 2004 [
      • Heesen C.
      • Kasper J.
      • Segal J.
      • Köpke S.
      • Mühlhauser I.
      Decisional role preferences, risk knowledge and information interests in patients with multiple sclerosis.
      ]
      ID
      RIKNO 1.0 questionnaire
       Heesen et al. 2015 [
      • Heesen C.
      • Kasper J.
      • Fischer K.
      • Köpke S.
      • Rahn A.
      • Backhus I.
      • et al.
      Risk knowledge in relapsing multiple sclerosis (RIKNO 1.0) - development of an outcome instrument for educational interventions.
      ]
      DDI
      RIKNO 2.0 questionnaire
       Heesen et al. 2017 [
      • Heesen C.
      • Pöttgen J.
      • Rahn A.C.
      • Liethmann K.
      • Kasper J.
      • Vahter L.
      • et al.
      What should a person with relapsing-remitting multiple sclerosis know? – focus group and survey data of a risk knowledge questionnaire (RIKNO 2.0).
      ]
      DD
       Giordano et al. 2018 [
      • Giordano A.
      • Liethmann K.
      • Köpke S.
      • Poettgen J.
      • Rahn A.C.
      • Drulovic J.
      • et al.
      Risk knowledge of people with relapsing-remitting multiple sclerosis – results of an international survey.
      ]
      DVD
      Unnamed 1
       Abolfazli et al. 2014 [
      • Abolfazli R.
      • Elyasi A.
      • Javadi M.R.
      • Gholami K.
      • Torkamandi H.
      • Amir-Shahkarami M.
      • et al.
      Knowledge and attitude assessment of Iranian multiple sclerosis patients receiving interferon beta.
      ]
      ID
      Unnamed 2
       Rath et al. 2017 [
      • Rath L.
      • Vijiaratnam N.
      • Skibina O.
      Assessing understanding of individual risk and symptoms of progressive multifocal leukoencephalopathy in patients prescribed natalizumab for multiple sclerosis: patient understanding: natalizumab-PML.
      ]
      D
      Abbreviations: V, very good; A, adequate; D, doubtful; I, inadequate; MSHLQ, Multiple Sclerosis Health Literacy Questionnaire; MSKQ, Multiple Sclerosis Knowledge Questionnaire; MSSM, Multiple Sclerosis Self-Management; PAM, Patient Activation Measure; RIKNO, Risk Knowledge.
      Empty cells indicate that the measurement property assessment was not performed.
      Table 5Summary of findings for each measurement instrument according to the recommendation for use
      Measurement instrumentMeasurement propertySummary of resultsOverall ratingQuality of evidence
      Category A: Measurment instruments whose content validity had sufficient evidence, and at least a low quality of evidence for a sufficient internal consistency of its scores (recommended for use)
       PAMContent validityNASufficientVery low: No content validity studies, measurement instrument development study inadequate validity, and study was performed in another population of interest
      Structural validityRash (unidimensionality): infit and outfit ranged from ≥0.5 to ≤1.5SufficientLow: There is one study of very good quality available, and study was performed in another population of interest
      Internal consistencyCronbach’s alpha 0.91; total sample size = 486SufficientLow: There is one study of very good quality available, and study was performed in another population of interest
      Measurement errorMIC not definedIndeterminate
      Hypothesis testing5 out of 6 hypotheses confirmedSufficientVery low: There is one study of doubtful quality available, and study was performed in another population of interest
       PAM short formContent validityNASufficientVery low: No content validity studies, measurement instrument development study inadequate validity, and only part of the study population consisted of patients with the disease of interest
      Structural validityRash (unidimensionality): infit and outfit ranged from ≥0.5 to ≤1.5SufficientLow: There is one study of very good quality available, inconsistency was found, and only part of the study population consisted of patients with the disease of interest
      Internal consistencyCronbach’s alpha 0.88; total sample size = 1,714SufficientModerate: There are two studies of very good quality available and only part of the study population consisted of patients with the disease of interest
      Hypothesis testing11 out of 11 hypotheses confirmedSufficientModerate: There is one study of very good quality available and only part of the study population consisted of patients with the disease of interest
      Category B: Measurement instruments with high-quality evidence for an insufficient measurement property (unrecommended for use)
       MSSM scale-revisedContent validityNASufficientModerate: At least one content validity study of doubtful quality
      Structural validityResults of CFAs are inconsistentInconsistent
      Internal consistencyCronbach’s alpha 0.59–0.91; total sample size = 1,616Indeterminate
      ReliabilityICC 0.64–0.88; total sample size = 31SufficientVery low: There is one study of adequate quality available, and the total sample included in the study is below 50
      Hypothesis testing3 out of 7 hypotheses confirmedInsufficientHigh: There is one study of very good quality available
       MSSM-2 scaleContent validityNASufficientVery low: No content validity studies and measurement instrument development study inadequate validity
      Structural validityCFA (7 factors): CFI = 0.91 and RMSEA = 0.05SufficientHigh: There is one study of very good quality available
      Internal consistencyCronbach’s alpha 0.54–0.89; total sample size = 1,197InsufficientHigh: There is one study of very good quality available
      Hypothesis testing1 out of 5 hypotheses confirmedInsufficientHigh: There is one study of very good quality available
      Category C: Measurement instruments categorized not in A or B (recommended for use until further evidence is provided)
       Comprehension of Confidence Intervals QuestionnaireContent validityNASufficientVery low: No content validity studies and measurement instrument development study inadequate validity
      Internal consistencyCronbach’s alpha 0.57–0.21; total sample size = 64Indeterminate
      Hypothesis testing0 out of 1 hypothesis confirmedInsufficientVery low: There is only one study of doubtful quality available, and the total sample included in the study is below 100
       Knowledge of Multiple Sclerosis ScaleContent validityNASufficientVery low: No content validity studies and measurement instrument development study inadequate validity
      Hypothesis testingThere is no information about resultsIndeterminate
       MSHLQContent validityNASufficientLow: No content validity studies and measurement instrument development study doubtful validity
      Structural validityEFA: 58% of explained varianceIndeterminate
      Internal consistencyCronbach’s alpha 0.84–0.97; total sample size = 210Indeterminate
      ReliabilityICC 0.88–0.96; total sample size = 20SufficientVery low: There is only one study of doubtful quality available, and the total sample included in the study is below 50
      Hypothesis testing1 out of 1 hypothesis confirmedSufficientLow: There is only one study of doubtful quality available
       MSKQContent validityNASufficientLow: No content validity studies and measurement instrument development study doubtful validity
      Internal consistencyKR20: 0.76; total sample size = 102Indeterminate
      Hypothesis testing2 out of 2 hypotheses confirmedSufficientLow: There is only one study of doubtful quality available
       MSSM scaleContent validityNASufficientVery low: No content validity studies and measurement instrument development study inadequate validity
      Structural validityEFA: 50% of explained varianceIndeterminate
      Internal consistencyCronbach’s alpha 0.84–0.87; total sample size = 554Indeterminate
      Hypothesis testing6 out of 8 hypotheses confirmedSufficientHigh: There are two studies of very good quality available
       RIKNO questionnaireContent validityNASufficientLow: No content validity studies and measurement instrument development study doubtful validity
      Hypothesis testing3 out 4 hypotheses confirmedSufficientLow: There is only one study of doubtful quality available
       RIKNO 1.0 questionnaireContent validityNASufficientLow: No content validity studies and measurement instrument development study doubtful validity
      Hypothesis testing0 out 10 hypotheses confirmedInsufficientLow: There is only one study of doubtful quality available
      ResponsivenessNo hypothesis definedIndeterminate
       RIKNO 2.0 questionnaireContent validityNASufficientLow (study on the relevance and comprehensiveness): No content validity studies and measurement instrument development study doubtful validity

      Moderate (study on the comprehensibility): At least one content validity study of doubtful quality
      Internal consistencyCronbach’s alpha 0.73; total sample size = 708Indeterminate
      Hypothesis testing4 out 4 hypotheses confirmedSufficientHigh: There is one study of very good quality available
       Unnamed 1Content validityNASufficientVery low: No content validity studies and measurement instrument development study inadequate validity
      Internal consistencyNot all information for “+” reported; total sample size = 20Indeterminate
      ReliabilityNot all information for “+” reported; total sample size = 20Indeterminate
       Unnamed 2Content validityNASufficientVery low: No content validity studies and measurement instrument development study inadequate validity
      Hypothesis testing5 out 7 hypotheses confirmedInsufficientVery low: There is only one study of doubtful quality available, and the total sample size included in the study is below 50
      Abbreviations: CFA, confirmatory factor analysis; CFI, comparative fit index; EFA, exploratory factor analysis; ICC, intraclass correlation coefficient; KR, Kuder–Richardson; MIC, minimal important change; MSHLQ, Multiple Sclerosis Health Literacy Questionnaire; MSKQ, Multiple Sclerosis Knowledge Questionnaire; MSSM, Multiple Sclerosis Self-Management; NA, not applicable; PAM, Patient Activation Measure; RIKNO, Risk Knowledge; RMSEA, root mean square error of approximation.

      3.1 Content validity

      The development quality of nine measurement instruments was considered inadequate. This is mainly due to aspects related to the participation of the representing target population in their development and in assessing their comprehension. Furthermore, the reports of the development of some of these instruments did not describe whether a pilot test had been performed. We rated their development as doubtful for the remaining five instruments because the methodological aspects were insufficiently described. The type of instrument version assessed (final or preliminary), the interviewers’ skills, the use of an interview guide, the approach used to analyze the data, and the number of researchers involved in the analysis were some of the unclear aspects.
      We found four studies that assessed content validity aspects for the Multiple Sclerosis Self-Management (MSSM) scale-revised and one that assessed them for the RIKNO (Risk Knowledge) 2.0 questionnaire (Table 4). Ghahari et al. [
      • Ghahari S.
      • Khoshbin L.S.
      • Forwell S.J.
      The multiple sclerosis self-management scale.
      ] assessed the relevance, comprehensiveness, and comprehensibility of the items of the MSSM scale-revised. However, the method used was not clearly described. The comprehensibility of Persian [
      • Saadat S.
      • Kajbaf M.B.
      • Kalantari M.
      • Hosseininezhad M.
      The multiple sclerosis self-management scale–revised (MSSM-R).
      ], Polish [
      • Tomczak M.
      • Kleka P.
      • Wilski M.
      Psychometric properties of the polish version of the multiple sclerosis self-management scale – revised.
      ], and Turkish [
      • Erbay Ö.
      • Usta Yeşilbalkan Ö.
      • Yüceyar N.
      • Baklan M.
      • Karadakovan A.
      • Tekindal M.A.
      Validity and reliability study of the Turkish version of multiple sclerosis self-management scale.
      ] versions of this instrument was also assessed, but either the target population did not participate in the assessment [
      • Erbay Ö.
      • Usta Yeşilbalkan Ö.
      • Yüceyar N.
      • Baklan M.
      • Karadakovan A.
      • Tekindal M.A.
      Validity and reliability study of the Turkish version of multiple sclerosis self-management scale.
      ,
      • Saadat S.
      • Kajbaf M.B.
      • Kalantari M.
      • Hosseininezhad M.
      The multiple sclerosis self-management scale–revised (MSSM-R).
      ], or the methodology was not clearly described [
      • Tomczak M.
      • Kleka P.
      • Wilski M.
      Psychometric properties of the polish version of the multiple sclerosis self-management scale – revised.
      ]. Regarding the RIKNO 2.0 questionnaire, the content validity study only assessed the comprehensibility of its translated versions.
      Thus, all instruments showed sufficient evidence for content validity. However, due to the low methodological quality of development studies, which were predominantly inadequate or doubtful, and the scarcity of content validity studies, the reviewers’ ratings mainly counted for the evidence synthesis, leading to very low or low quality evidence of sufficient content validity for most instruments (Table 5).

      3.2 Construct validity

      We did not identify any studies that assessed the structural validity of eight instruments. The structural validity of two instruments [Multiple Sclerosis Health Literacy Questionnaire (MSHLQ) and MSSM scale] was rated as indeterminate because the identified studies did not provide sufficient information. Findings on the structural validity of the MSSM scale-revised were inconsistent. The confirmatory factor analysis of a five-factor structure showed a good model fit in one study [
      • Saadat S.
      • Kajbaf M.B.
      • Kalantari M.
      • Hosseininezhad M.
      The multiple sclerosis self-management scale–revised (MSSM-R).
      ] and a poor model fit in another [
      • Tomczak M.
      • Kleka P.
      • Wilski M.
      Psychometric properties of the polish version of the multiple sclerosis self-management scale – revised.
      ] (Supplementary Table A). Only three instruments showed sufficient evidence for structural validity: the MSSM-2 scale, the Patient Activation Measure (PAM), and the PAM short form (Table 5). Hypothesis testing for construct validity was assessed in 13 instruments. The results of the convergent validity and discriminative validity tests are described in Supplementary Table A. Seven instruments showed sufficient evidence, five showed insufficient evidence for this measurement property, and one could not be rated due to a lack of information. We did not find any study that assessed cross-cultural validity aspects.

      3.3 Criterion validity

      None of the included studies reported a comparison of a shortened instrument with its original long version.

      3.4 Reliability

      The internal consistency of the scores was assessed in nine instruments (Supplementary Table A and Table 5). However, we could only determine the results of three instruments as the rest presented insufficient evidence on their structural validity. The PAM and the PAM short form showed sufficient evidence for this measurement property with low and moderate quality, respectively. In contrast, the MSSM-2 scale showed insufficient evidence with a high degree of quality. Test–retest reliability of scores was assessed in three instruments. The MSHLQ and the MSSM scale-revised showed sufficient evidence for this measurement property, but both with very low quality (Table 5). Regarding the instrument developed by Abolfazli et al. [
      • Abolfazli R.
      • Elyasi A.
      • Javadi M.R.
      • Gholami K.
      • Torkamandi H.
      • Amir-Shahkarami M.
      • et al.
      Knowledge and attitude assessment of Iranian multiple sclerosis patients receiving interferon beta.
      ], the available evidence for this property could not be rated because they did not provide information on the observed intraclass correlation coefficients. Measurement error was only assessed in the PAM short form, but we could not interpret this given the unavailability of information on the minimal important change.

      3.5 Responsiveness

      The responsiveness of scores was assessed only in the RIKNO 1.0 questionnaire (Supplementary Table A and Table 5). However, the statistical significance of the change was assessed rather than testing hypotheses about expected differences in changes between the groups.

      3.6 Categorization of measurement instruments

      Based on the findings, we classified two instruments as A, the PAM and the PAM short form, and these can be recommended for use. The MSSM scale-revised and the MSSM-2 scale were classified as B, so their use cannot be recommended. The remaining 10 instruments were categorized as C; the quality of the content validity evidence for 5 of them was higher than the others [the MSHLQ, the Multiple Sclerosis Knowledge Questionnaire (MSKQ), and the three RIKNO questionnaire versions] (Table 5), and they could be provisionally recommended for use until further evidence is provided. The RIKNO questionnaire versions have the fewest number of items; however, a certain level of numeracy may be required from participants to answer it (Supplementary Table B). The length of the MSHLQ and the MSKQ is similar, although the completion time was shorter in the case of the MSHLQ; furthermore, the instruments’ scores were positively associated with the educational level of the participants.

      4. Discussion

      This review was designed to determine the most suitable measurement instruments of disease-related knowledge of people with MS. Its findings show that only two instruments can be recommended for use, and five could be provisionally recommended until further evidence is provided.
      Comprehensive database searches and the use of a rigorous and innovative methodology are key strengths of this review. However, subjectivity may have affected review processes. Studies were reviewed independently to mitigate this potential limitation and ratings were agreed upon by consensus among the review team to reduce interpretation variability. In addition, psychometric reviews are complex as they involve multiple reviews, one for each measurement property. Consequently, the review team included reviewers with knowledge of the construct of interest and experience with the target population and with the field of psychometrics and qualitative research. Finally, public and patient involvement in research is an increasingly important issue [
      • Domecq J.P.
      • Prutsky G.
      • Elraiyah T.
      • Wang Z.
      • Nabhan M.
      • Shippee N.
      • et al.
      Patient engagement in research: a systematic review.
      ]. Therefore, patient involvement in future reviews would probably have to be considered, particularly in assessing the content validity of the instruments.
      We found that the PAM and the PAM short form were the most suitable instruments. Both measure “activation,” understanding this construct as knowledge, skills, and confidence in self-management of one’s health or chronic condition [
      • Hibbard J.H.
      • Stockard J.
      • Mahoney E.R.
      • Tusler M.
      Development of the patient activation measure (PAM): conceptualizing and measuring activation in patients and consumers: development of the patient activation measure (PAM).
      ,
      • Hibbard J.H.
      • Mahoney E.R.
      • Stockard J.
      • Tusler M.
      Development and testing of a short form of the patient activation measure.
      ]. Therefore, they do not strictly measure knowledge but consider it a subconstruct of the “activation.” In addition, both are generic instruments, not specifically targeted at people with MS, although they have been used in studies in the area of MS [
      • Wilkie D.D.
      • Solari A.
      • Nicholas R.S.J.
      The impact of the face-to-face consultation on decisional conflict in complex decision-making in multiple sclerosis: a pilot study.
      ,
      • Rahn A.C.
      • Wenzel L.
      • Icks A.
      • Stahmann A.
      • Scheiderbauer J.
      • Grentzenberg K.
      • et al.
      Development and evaluation of an interactive web-based decision-making programme on relapse management for people with multiple sclerosis ([email protected])—study protocol for a randomised controlled trial.
      ]. However, researchers and clinicians should consider other instruments to perform more specific measures of MS-related knowledge.
      Among these specific instruments, we provisionally recommended the use of five until more evidence is available: the MSHLQ, the MSKQ, and the three RIKNO questionnaire versions. The quality of the evidence for their content validity was rated as low. We did not identify any content validity studies of these five instruments other than one conducted by Giordano et al. [
      • Giordano A.
      • Liethmann K.
      • Köpke S.
      • Poettgen J.
      • Rahn A.C.
      • Drulovic J.
      • et al.
      Risk knowledge of people with relapsing-remitting multiple sclerosis – results of an international survey.
      ] to assess the comprehensibility of the RIKNO 2.0 questionnaire translation. Furthermore, we only identified a single study that assessed four of these instruments [
      • Dehghani A.
      • Keshavarzi A.
      Development and validation of a multidimensional health literacy questionnaire for multiple sclerosis patients.
      ,
      • Giordano A.
      • Messmer Uccelli M.
      • Pucci E.
      • Martinelli V.
      • Borreani C.
      • Lugaresi A.
      • et al.
      The Multiple Sclerosis Knowledge Questionnaire: a self-administered instrument for recently diagnosed patients.
      ,
      • Heesen C.
      • Kasper J.
      • Segal J.
      • Köpke S.
      • Mühlhauser I.
      Decisional role preferences, risk knowledge and information interests in patients with multiple sclerosis.
      ,
      • Heesen C.
      • Kasper J.
      • Fischer K.
      • Köpke S.
      • Rahn A.
      • Backhus I.
      • et al.
      Risk knowledge in relapsing multiple sclerosis (RIKNO 1.0) - development of an outcome instrument for educational interventions.
      ] and two studies that assessed the RIKNO 2.0 version [
      • Heesen C.
      • Pöttgen J.
      • Rahn A.C.
      • Liethmann K.
      • Kasper J.
      • Vahter L.
      • et al.
      What should a person with relapsing-remitting multiple sclerosis know? – focus group and survey data of a risk knowledge questionnaire (RIKNO 2.0).
      ,
      • Giordano A.
      • Liethmann K.
      • Köpke S.
      • Poettgen J.
      • Rahn A.C.
      • Drulovic J.
      • et al.
      Risk knowledge of people with relapsing-remitting multiple sclerosis – results of an international survey.
      ]. Given that measurement instruments are used with diverse groups of people and in different circumstances, further evidence is needed to assess whether they are valid and reliable for such use [
      • Prinsen C.A.C.
      • Mokkink L.B.
      • Bouter L.M.
      • Alonso J.
      • Patrick D.L.
      • de Vet H.C.W.
      • et al.
      COSMIN guideline for systematic reviews of patient-reported outcome measures.
      ,
      • Terwee C.B.
      • Prinsen C.A.C.
      • Chiarotto A.
      • Westerman M.J.
      • Patrick D.L.
      • Alonso J.
      • et al.
      COSMIN methodology for evaluating the content validity of patient-reported outcome measures: a Delphi study.
      ].
      According to the COSMIN guidelines, a measurement instrument can be recommended for use if, in addition to sufficient evidence of content validity, it shows sufficient internal consistency of at least low quality evidence [
      • Prinsen C.A.C.
      • Mokkink L.B.
      • Bouter L.M.
      • Alonso J.
      • Patrick D.L.
      • de Vet H.C.W.
      • et al.
      COSMIN guideline for systematic reviews of patient-reported outcome measures.
      ,
      • Mokkink L.B.
      • de Vet H.C.W.
      • Prinsen C.A.C.
      • Patrick D.L.
      • Alonso J.
      • Bouter L.M.
      • et al.
      COSMIN risk of bias checklist for systematic reviews of patient-reported outcome measures.
      ,
      • Terwee C.B.
      • Prinsen C.A.C.
      • Chiarotto A.
      • Westerman M.J.
      • Patrick D.L.
      • Alonso J.
      • et al.
      COSMIN methodology for evaluating the content validity of patient-reported outcome measures: a Delphi study.
      ]. Sufficient evidence of the structural validity of the measurement instrument is needed to be able to interpret the internal consistency coefficients [
      • Prinsen C.A.C.
      • Mokkink L.B.
      • Bouter L.M.
      • Alonso J.
      • Patrick D.L.
      • de Vet H.C.W.
      • et al.
      COSMIN guideline for systematic reviews of patient-reported outcome measures.
      ]. Therefore, the evidence on the internal consistency of seven instruments was rated as indeterminate due to the lack of evidence of this measurement property. Furthermore, analysis of the internal structure is only relevant when the instrument is based on a reflective model that assumes all items of a scale or subscale are manifestations of an underlying construct [
      • Avila M.L.
      • Stinson J.
      • Kiss A.
      • Brandão L.R.
      • Uleryk E.
      • Feldman B.M.
      A critical review of scoring options for clinical measurement tools.
      ]. None of the included studies described the type of model on which they were based. Consequently, according to COSMIN’s recommendations, we considered all the identified instruments to be based on a reflective model and interpreted the analyses of their internal structure [
      • Mokkink L.B.
      • de Vet H.C.W.
      • Prinsen C.A.C.
      • Patrick D.L.
      • Alonso J.
      • Bouter L.M.
      • et al.
      COSMIN risk of bias checklist for systematic reviews of patient-reported outcome measures.
      ]. In future analyses of these instruments and the development of new ones, it would be desirable to report whether instruments are based on reflective or formative models to justify the relevance of structural validity analysis.
      The most studied measurement property, and for which seven of the instruments showed sufficient evidence, was hypothesis testing for construct validity. On the other hand, the test–retest reliability was one of the least studied: only three instruments performed such an assessment, and only two presented sufficient evidence, which was of very low quality. Concerning cross-cultural validity, although many original versions have been translated into other languages or adapted to other cultures, we have not identified any studies that have assessed this. Such studies are necessary to assess whether measures from one population of a given culture are equivalent to those from another population with different cultural characteristics [
      • Mokkink L.B.
      • de Vet H.C.W.
      • Prinsen C.A.C.
      • Patrick D.L.
      • Alonso J.
      • Bouter L.M.
      • et al.
      COSMIN risk of bias checklist for systematic reviews of patient-reported outcome measures.
      ].
      Based on the available evidence, only 2 out of 14 disease-related knowledge measurement instruments are suitable. However, these two instruments assess a broader construct than knowledge and are not explicitly aimed at people with MS. Five instruments could potentially be recommended for use among the identified instruments that strictly measure MS-related knowledge. Nevertheless, further research is required to examine their suitability more closely. This review identifies evidence gaps in the available measurement instruments and thus provides a helpful framework for both new assessments of these instruments and the development of new ones. Review findings will also help researchers and clinicians make evidence-based decisions about the use of these measurement instruments.

      Acknowledgment

      The authors appreciate the review of the English text by Mr Andy Hughes.

      References

        • Thompson A.J.
        • Baranzini S.E.
        • Geurts J.
        • Hemmer B.
        • Ciccarelli O.
        Multiple sclerosis.
        Lancet. 2018; 391: 1622-1636
        • Filippi M.
        • Bar-Or A.
        • Piehl F.
        • Preziosa P.
        • Solari A.
        • Vukusic S.
        • et al.
        Multiple sclerosis.
        Nat Rev Dis Primer. 2018; 4: 43
        • Wingerchuk D.M.
        • Weinshenker B.G.
        Disease modifying therapies for relapsing multiple sclerosis.
        BMJ. 2016; 354: i3518
        • Bokhour B.G.
        • Fix G.M.
        • Mueller N.M.
        • Barker A.M.
        • Lavela S.L.
        • Hill J.N.
        • et al.
        How can healthcare organizations implement patient-centered care? Examining a large-scale cultural transformation.
        BMC Health Serv Res. 2018; 18: 168
        • Heesen C.
        • Köpke S.
        • Solari A.
        • Geiger F.
        • Kasper J.
        Patient autonomy in multiple sclerosis — possible goals and assessment strategies.
        J Neurol Sci. 2013; 331: 2-9
        • Colligan E.
        • Metzler A.
        • Tiryaki E.
        Shared decision-making in multiple sclerosis.
        Mult Scler J. 2017; 23: 185-190
      1. Communication theory and health behavior change.
        in: Glanz K. Rimer B.K. Viswanath K. Health behavior and health education: theory, research, and practice. 4th ed. Jossey-Bass, San Francisco, CA2008
        • Vermersch P.
        • Shanahan J.
        • Langdon D.
        • Yeandle D.
        • Alexandri N.
        • Schippling S.
        Knowledge is power, but is ignorance bliss? Optimising conversations about disease progression in multiple sclerosis.
        Neurol Ther. 2020; 9: 1-10
        • Coulter A.
        • Ellins J.
        Effectiveness of strategies for informing, educating, and involving patients.
        BMJ. 2007; 335: 24-27
        • Köpke S.
        • Solari A.
        • Rahn A.
        • Khan F.
        • Heesen C.
        • Giordano A.
        Information provision for people with multiple sclerosis. Cochrane multiple sclerosis and rare diseases of the CNS group.
        Cochrane Database Syst Rev. 2018; : CD008757
        • Prinsen C.A.C.
        • Mokkink L.B.
        • Bouter L.M.
        • Alonso J.
        • Patrick D.L.
        • de Vet H.C.W.
        • et al.
        COSMIN guideline for systematic reviews of patient-reported outcome measures.
        Qual Life Res. 2018; 27: 1147-1157
        • Mokkink L.B.
        • de Vet H.C.W.
        • Prinsen C.A.C.
        • Patrick D.L.
        • Alonso J.
        • Bouter L.M.
        • et al.
        COSMIN risk of bias checklist for systematic reviews of patient-reported outcome measures.
        Qual Life Res. 2018; 27: 1171-1179
        • Terwee C.B.
        • Prinsen C.A.C.
        • Chiarotto A.
        • Westerman M.J.
        • Patrick D.L.
        • Alonso J.
        • et al.
        COSMIN methodology for evaluating the content validity of patient-reported outcome measures: a Delphi study.
        Qual Life Res. 2018; 27: 1159-1170
        • Page M.J.
        • McKenzie J.E.
        • Bossuyt P.M.
        • Boutron I.
        • Hoffmann T.C.
        • Mulrow C.D.
        • et al.
        The PRISMA 2020 statement: an updated guideline for reporting systematic reviews.
        BMJ. 2021; 372: n71
        • Rethlefsen M.L.
        • Kirtley S.
        • Waffenschmidt S.
        • Waffenschmidt S.
        • Ayala A.P.
        • Moher D.
        • et al.
        PRISMA-S: an extension to the PRISMA statement for reporting literature searches in systematic reviews.
        Syst Rev. 2021; 10: 39
        • Terwee C.B.
        • Jansma E.P.
        • Riphagen I.I.
        • de Vet H.C.W.
        Development of a methodological PubMed search filter for finding studies on measurement properties of measurement instruments.
        Qual Life Res. 2009; 18: 1115-1123
        • Ouzzani M.
        • Hammady H.
        • Fedorowicz Z.
        • Elmagarmid A.
        Rayyan—a web and mobile app for systematic reviews.
        Syst Rev. 2016; 5: 210
      2. COnsensus-based Standards for the selection of health Measurement INstruments. COSMIN tools. COSMIN.
        (Available at)
        • Gonzalez-del-Rio M.
        • Bertran-Noguer C.
        • Ramió-Torrentà L.
        • Zabaleta-del-Olmo E.
        Disease-related knowledge measurement instruments of people affected by multiple sclerosis: protocol for a systematic psychometric review.
        BMJ Open. 2021; 11: e049580
        • Rahn A.C.
        • Backhus I.
        • Fuest F.
        • Riemann-Lorenz K.
        • Köpke S.
        • van de Roemer A.
        • et al.
        Comprehension of confidence intervals - development and piloting of patient information materials for people with multiple sclerosis: qualitative study and pilot randomised controlled trial.
        BMC Med Inform Decis Mak. 2016; 16: 122
        • Maybury C.P.
        • Brewin C.R.
        Social relationships, knowledge and adjustment to multiple sclerosis.
        J Neurol Neurosurg Psychiatry. 1984; 47: 372-376
        • Dehghani A.
        • Keshavarzi A.
        Development and validation of a multidimensional health literacy questionnaire for multiple sclerosis patients.
        Mult Scler Relat Disord. 2018; 25: 156-162
        • Giordano A.
        • Messmer Uccelli M.
        • Pucci E.
        • Martinelli V.
        • Borreani C.
        • Lugaresi A.
        • et al.
        The Multiple Sclerosis Knowledge Questionnaire: a self-administered instrument for recently diagnosed patients.
        Mult Scler. 2010; 16: 100-111
        • Bishop M.
        • Frain M.
        Development and initial analysis of multiple sclerosis self-management scale.
        Int J MS Care. 2007; 9: 35-42
        • Bishop M.
        • Frain M.P.
        • Tschopp M.K.
        Self-management, perceived control, and subjective quality of life in multiple sclerosis: an exploratory study.
        Rehabilitation Counseling Bulletin. 2008; 52: 45-56
        • Bishop M.
        • Frain M.P.
        • Rumrill P.D.
        • Rymond C.
        The relationship of self-management and disease modifying therapy use to employment status among adults with multiple sclerosis.
        J Vocat Rehabil. 2009; 31: 119-127
        • Bishop M.
        • Frain M.P.
        The multiple sclerosis self-management scale: revision and psychometric analysis.
        Rehabil Psychol. 2011; 56: 150-159
        • Ghahari S.
        • Khoshbin L.S.
        • Forwell S.J.
        The multiple sclerosis self-management scale.
        Int J MS Care. 2014; 16: 61-67
        • Wilski M.
        • Tasiemski T.
        • Kocur P.
        Demographic, socioeconomic and clinical correlates of self-management in multiple sclerosis.
        Disabil Rehabil. 2015; 37: 1970-1975
        • Wilski M.
        • Tasiemski T.
        Illness perception, treatment beliefs, self-esteem, and self-efficacy as correlates of self-management in multiple sclerosis.
        Acta Neurol Scand. 2016; 133: 338-345
        • Erbay Ö.
        • Usta Yeşilbalkan Ö.
        • Yüceyar N.
        • Baklan M.
        • Karadakovan A.
        • Tekindal M.A.
        Validity and reliability study of the Turkish version of multiple sclerosis self-management scale.
        J Neurosci Nurs. 2020; 52: 122-127
        • Saadat S.
        • Kajbaf M.B.
        • Kalantari M.
        • Hosseininezhad M.
        The multiple sclerosis self-management scale–revised (MSSM-R).
        Int J MS Care. 2020; 22: 37-42
        • Tomczak M.
        • Kleka P.
        • Wilski M.
        Psychometric properties of the polish version of the multiple sclerosis self-management scale – revised.
        Disabil Rehabil. 2020; 44: 2113-2122
        • Bishop M.L.
        • Frain M.P.
        • Li J.
        • Chiu C.Y.
        • McDaniels B.
        • Kim B.J.
        The multiple sclerosis self-management scale-2: evaluation of an updated scale.
        J Appl Rehabil Couns. 2019; 50: 210-226
        • Hibbard J.H.
        • Stockard J.
        • Mahoney E.R.
        • Tusler M.
        Development of the patient activation measure (PAM): conceptualizing and measuring activation in patients and consumers: development of the patient activation measure (PAM).
        Health Serv Res. 2004; 39: 1005-1026
        • Hibbard J.H.
        • Mahoney E.R.
        • Stockard J.
        • Tusler M.
        Development and testing of a short form of the patient activation measure.
        Health Serv Res. 2005; 40: 1918-1930
        • Stepleman L.
        • Rutter M.C.
        • Hibbard J.
        • Johns L.
        • Wright D.
        • Hughes M.
        Validation of the patient activation measure in a multiple sclerosis clinic sample and implications for care.
        Disabil Rehabil. 2010; 32: 1558-1567
        • Goodworth M.C.R.
        • Stepleman L.
        • Hibbard J.
        • Johns L.
        • Wright D.
        • Hughes M.D.
        • et al.
        Variables associated with patient activation in persons with multiple sclerosis.
        J Health Psychol. 2016; 21: 82-92
        • Heesen C.
        • Kasper J.
        • Segal J.
        • Köpke S.
        • Mühlhauser I.
        Decisional role preferences, risk knowledge and information interests in patients with multiple sclerosis.
        Mult Scler J. 2004; 10: 643-650
        • Heesen C.
        • Kasper J.
        • Fischer K.
        • Köpke S.
        • Rahn A.
        • Backhus I.
        • et al.
        Risk knowledge in relapsing multiple sclerosis (RIKNO 1.0) - development of an outcome instrument for educational interventions.
        PLoS One. 2015; 10: e0138364
        • Heesen C.
        • Pöttgen J.
        • Rahn A.C.
        • Liethmann K.
        • Kasper J.
        • Vahter L.
        • et al.
        What should a person with relapsing-remitting multiple sclerosis know? – focus group and survey data of a risk knowledge questionnaire (RIKNO 2.0).
        Mult Scler Relat Disord. 2017; 18: 186-195
        • Giordano A.
        • Liethmann K.
        • Köpke S.
        • Poettgen J.
        • Rahn A.C.
        • Drulovic J.
        • et al.
        Risk knowledge of people with relapsing-remitting multiple sclerosis – results of an international survey.
        PLoS One. 2018; 13: e0208004
        • Abolfazli R.
        • Elyasi A.
        • Javadi M.R.
        • Gholami K.
        • Torkamandi H.
        • Amir-Shahkarami M.
        • et al.
        Knowledge and attitude assessment of Iranian multiple sclerosis patients receiving interferon beta.
        Iran J Neurol. 2014; 13: 160-167
        • Rath L.
        • Vijiaratnam N.
        • Skibina O.
        Assessing understanding of individual risk and symptoms of progressive multifocal leukoencephalopathy in patients prescribed natalizumab for multiple sclerosis: patient understanding: natalizumab-PML.
        Intern Med J. 2017; 47: 194-199
        • Domecq J.P.
        • Prutsky G.
        • Elraiyah T.
        • Wang Z.
        • Nabhan M.
        • Shippee N.
        • et al.
        Patient engagement in research: a systematic review.
        BMC Health Serv Res. 2014; 14: 89
        • Wilkie D.D.
        • Solari A.
        • Nicholas R.S.J.
        The impact of the face-to-face consultation on decisional conflict in complex decision-making in multiple sclerosis: a pilot study.
        Mult Scler J Exp Transl Clin. 2020; 6 (205521732095980)
        • Rahn A.C.
        • Wenzel L.
        • Icks A.
        • Stahmann A.
        • Scheiderbauer J.
        • Grentzenberg K.
        • et al.
        Development and evaluation of an interactive web-based decision-making programme on relapse management for people with multiple sclerosis ([email protected])—study protocol for a randomised controlled trial.
        Trials. 2021; 22: 139
        • Avila M.L.
        • Stinson J.
        • Kiss A.
        • Brandão L.R.
        • Uleryk E.
        • Feldman B.M.
        A critical review of scoring options for clinical measurement tools.
        BMC Res Notes. 2015; 8: 612