Advertisement

No difference in knowledge obtained from infographic or plain language summary of a Cochrane systematic review: three randomized controlled trials

Open AccessPublished:December 18, 2017DOI:https://doi.org/10.1016/j.jclinepi.2017.12.003

      Abstract

      Objectives

      The aim of this study was to test the usefulness of an infographic in the translation of knowledge about health information from a Cochrane systematic review to lay and professional populations in comparison to a plain language summary (PLS) and scientific abstract (SA).

      Study Design and Setting

      We conducted three parallel randomized trials with university students (n = 171), consumers (n = 99), and doctors (n = 64), to examine the effect of different summary formats of a Cochrane systematic review on the knowledge about health information presented in the review, reading experience, and perceived user-friendliness. In the trials involving students and doctors, an infographic was compared to a PLS and a SA, while in those with consumers, an infographic was compared to a PLS.

      Results

      We found no difference in knowledge between the infographic and the text-based PLS in any of the trials or in the whole participant sample. All three participant groups preferred the infographic and gave it higher ratings for reading experience (d = 0.48 in the overall sample) and user-friendliness (d = 0.46 in the overall sample).

      Conclusion

      Although the infographic format was perceived as more enjoyable for reading, it was not better than a traditional, text-based PLS in the translation of knowledge about findings from a Cochrane systematic review.

      Keywords

      What is new?

        Key findings

      • For students, consumers, and doctors, there was no difference in knowledge about health from a Cochrane systematic review when it was presented as an infographic or a plain language summary.
      • The infographic format was perceived as more enjoyable for reading and more user-friendly than a written format.

        What this study adds to what was known?

      • Visual and textual summaries of information about health research may be equally effective in terms of information transfer to different audiences, but users may perceive their usefulness differently.

        What is the implication and what should change now?

      • These findings may inform Cochrane and other organizations about the best ways to present health information to patients and the public.
      • Translation of information from systematic reviews to different audiences should focus on examining factors underlying reading experiences in order to develop formats most suitable for capturing readers' attention and accelerating the knowledge translation process.

      1. Background

      Scientists are increasingly encouraged to “translate” their findings for the general public, and this is especially important for clinical research so that patients can be properly involved in decision-making. However, the best way of presenting research results to different audiences is not clear, as there have been relatively few studies on this topic and they indicate that scientific findings are not always easy to interpret [
      • Kurtzman E.T.
      • Greene J.
      Effective presentation of health care performance information for consumer decision making: a systematic review.
      ,
      • Maguire L.K.
      • Clarke M.
      How much do you need: a randomised experiment of whether readers can understand the key messages from summaries of Cochrane Reviews without reading the full review.
      ,
      • Alderdice F.
      • McNeill J.
      • Lasserson T.
      • Beller E.
      • Carroll M.
      • Hundley V.
      • et al.
      Do Cochrane summaries help student midwives understand the findings of Cochrane systematic reviews: the BRIEF randomised trial.
      ]. A recent systematic review [
      • Gagliardi A.R.
      • Legare F.
      • Brouwers M.C.
      • Webster F.
      • Badley E.
      • Straus S.
      Patient-mediated knowledge translation (PKT) interventions for clinical encounters: a systematic review.
      ] showed that there was no clear evidence for the best type of intervention for health care information translation. General recommendations are that recognizable graphics and plain language should be used in information presentation, that the information should be concise, and that new formats should always be tested before use [
      • Kurtzman E.T.
      • Greene J.
      Effective presentation of health care performance information for consumer decision making: a systematic review.
      ].
      The presentation of research evidence to lay audiences is heavily emphasized by Cochrane, which produces systematic reviews to summarize available evidence about medical questions [
      • Langendam M.W.
      • Akl E.A.
      • Dahm P.
      • Glasziou P.
      • Guyatt G.
      • Schunemann H.J.
      Assessing and presenting summaries of evidence in Cochrane Reviews.
      ]. In a Cochrane systematic review, key messages are presented in three ways: as a scientific abstract (SA), as a summary of findings (SoF) table aimed at researchers and health practitioners, and as a plain language summary (PLS) aimed at the lay public [
      • Santesso N.
      • Rader T.
      • Nilsen E.S.
      • Glenton C.
      • Rosenbaum S.
      • Ciapponi A.
      • et al.
      A summary to communicate evidence from systematic reviews to the public improved understanding and accessibility of information: a randomized controlled trial.
      ]. Although the language of the PLS should be simplified and modified for lay audiences, previous research has shown that PLS are generally no better than SA in terms of information uptake [
      • Maguire L.K.
      • Clarke M.
      How much do you need: a randomised experiment of whether readers can understand the key messages from summaries of Cochrane Reviews without reading the full review.
      ,
      • Alderdice F.
      • McNeill J.
      • Lasserson T.
      • Beller E.
      • Carroll M.
      • Hundley V.
      • et al.
      Do Cochrane summaries help student midwives understand the findings of Cochrane systematic reviews: the BRIEF randomised trial.
      ]. Furthermore, PLSs often do not actually follow Cochrane Standards for the reporting of Plain Language Summaries (PLEACS) [
      • Jelicic Kadic A.
      • Fidahic M.
      • Vujcic M.
      • Saric F.
      • Propadalo I.
      • Marelja I.
      • et al.
      Cochrane plain language summaries are highly heterogeneous with low adherence to the standards.
      ].
      Recently, Cochrane has introduced infographics to present the results of systematic reviews [http://visuallycochrane.net/]. An infographic provides short textual information supported by visual representations. The current evidence for the use of an infographic in comparison to other formats of health information presentation is inconsistent [
      • Comello M.L.
      • Qian X.
      • Deal A.M.
      • Ribisl K.M.
      • Linnan L.A.
      • Tate D.F.
      Impact of game-inspired infographics on user engagement and information processing in an eHealth program.
      ,
      • Crick K.
      • Hartling L.
      Preferences of knowledge users for two formats of summarizing results from systematic reviews: infographics and critical appraisals.
      ,
      • Arcia A.
      • Suero-Tejeda N.
      • Bales M.E.
      • Merrill J.A.
      • Yoon S.
      • Woollen J.
      • et al.
      Sometimes more is more: iterative participatory design of infographics for engagement of community members with varying levels of health literacy.
      ]. There are indications that the optimal type of format depends on the audience, for example, professionals have been shown to prefer text-based information, while consumers were more comfortable with an infographic [
      • Crick K.
      • Hartling L.
      Preferences of knowledge users for two formats of summarizing results from systematic reviews: infographics and critical appraisals.
      ]. It has been suggested that infographics should be information rich, placed in a context relevant to the reader, and should yield an accurate meaning even if interpreted literally [
      • Arcia A.
      • Suero-Tejeda N.
      • Bales M.E.
      • Merrill J.A.
      • Yoon S.
      • Woollen J.
      • et al.
      Sometimes more is more: iterative participatory design of infographics for engagement of community members with varying levels of health literacy.
      ].
      The aim of our study was to compare information presentation in an infographic with that in standard PLS and SA formats, in terms of gaining knowledge about research results presented in a Cochrane systematic review. We tested different summary formats in three groups of users: (1) university students, (2) consumers to whom the topic may be personally relevant, and (3) doctors.

      2. Methods

      2.1 Trial design

      We conducted three randomized controlled trials to test potential differences in knowledge, reading experience, and user-friendliness between infographic, PLS, and SA formats. The content of the formats was based on the same systematic review on external cephalic version for breech presentation at term [
      • Hofmeyr G.J.
      • Kulier R.
      • West H.M.
      External cephalic version for breech presentation at term.
      ], and the presentation modes were (1) visual presentation in plain language (infographic), (2) PLS written in accordance with the PLEACS standards [
      • Cochrane
      Standards for the reporting of Plain Language Summaries in new Cochrane Intervention Reviews (PLEACS).
      ] by a professional writer/editor and author in this study (E.W.), and (3) SA written by the authors of the systematic review. All three formats are publicly available in the Cochrane Library [
      • Hofmeyr G.J.
      • Kulier R.
      • West H.M.
      External cephalic version for breech presentation at term.
      ]. The intervention was delivered in an online survey, which consisted of five parts: (1) demographic data, (2) presentation of a randomly assigned summary format, (3) knowledge test on the information given in the summary, (4) questions about perceived user-friendliness of information presentation and overall reading experience with the summary format, and (5) health numeracy test [
      • Osborn C.Y.
      • Wallston K.A.
      • Shpigel A.
      • Cavanaugh K.
      • Kripalani S.
      • Rothman R.L.
      Development and validation of the general health numeracy test (GHNT).
      ].

      2.2 Participants

      2.2.1 Trial 1—students

      The setting of the trial was performed in the University of Split at the beginning of the 2016–2017 academic year (October 2016). Students attending the first year Medical Humanities course at the School of Medicine and students attending a course in psychology at the School of Humanities and Social Sciences were invited to participate in the trial. Medical students attended their course according to the group assignment published on the course website. Humanities students arrived in three different groups, according to the time they were scheduled to have a psychology course. Participation was voluntary and anonymous.

      2.2.2 Trial 2—consumers

      The trial was performed in November 2016. Eligible participants were female members of the Association for the Promotion of Patients' Rights (http://www.pravapacijenata.hr/hr/) and of the RODA Parents in Action consumer group for pregnancy and parenting (http://www.RODA.hr), which involves pregnant women, young mothers, and parenting and pregnancy advisors. The managers of the two consumer groups sent the survey to their members via e-mail and Facebook. The survey was anonymous and was open for 2 weeks.

      2.2.3 Trial 3—doctors

      Eligible participants were medical doctors in any specialty or field of work. Two authors (I.B. and L.P.) made a list of publicly available e-mail addresses of doctors–clinical researchers from the School of Medicine and University Hospital Center in Split. Authors also contacted colleagues at the university hospitals in Zagreb and asked them to distribute the web-link to the anonymous survey. The responses were collected in two waves in January–February 2017, lasting a total of 6 weeks.

      2.3 Intervention

      The infographic format of a Cochrane systematic review summary represented the experimental intervention in the trial. The control groups were presented with the PLS (in all three trials) or the SA (students and doctors). The intervention was delivered via the Survey Monkey platform (https://www.surveymonkey.com/).
      All materials were in Croatian (see English translation in Additional File 1/Appendix at www.jclinepi.com). All three types of summary (infographic, PLS, and SA) were translated into Croatian by the authors and back translated to English by an independent language expert to ensure the reliability of translation. There were no changes in back translation that would have affected the meaning of the text.

      2.3.1 Trial 1

      After the students had read the assigned summary format, they were asked to close computer monitors and continue with the course teaching. The course activity consisted of watching a 20-minute video related to the topic of the course (a TED video about Coursera, http://bit.ly/1zsKwX3). This neutral content was used to distract students from the text they had just read, in order to examine the amount of information retained from the summary format. We used a delayed testing method because it has been shown that, although immediate knowledge after tests can show significant improvement, such knowledge is often lost rapidly with time [
      • Bell D.S.
      • Harless C.E.
      • Higa J.K.
      • Bjork E.L.
      • Bjork R.A.
      • Bazargan M.
      • et al.
      Knowledge retention after an online tutorial: a randomized educational experiment among resident physicians.
      ].
      At the end of their course activity, the participants were invited to complete a questionnaire about the summary format. The participants were not allowed to use the Internet to get more information about the topic, converse with other students, or use their cell phones.

      2.3.2 Trials 2 and 3

      The participants were asked to read one of the summary formats, followed by the survey, which started with a numeracy test to ensure sufficient delay so that the knowledge test was not completed immediately after reading the summary. The consumers in trial 2 were presented with either the infographic or the PLS, and the doctors in trial 3 were allocated to one of the three formats. The SA format was not presented to consumers because that format is not intended for use by lay populations. The return page option was disabled in the online system so the respondents were not able to reread the summary; the same person could not retake the survey from the same IP address. There were no time limits for reading the format or answering the questionnaire.

      2.4 Outcomes

      2.4.1 Primary outcome

      In previous research [
      • Santesso N.
      • Rader T.
      • Nilsen E.S.
      • Glenton C.
      • Rosenbaum S.
      • Ciapponi A.
      • et al.
      A summary to communicate evidence from systematic reviews to the public improved understanding and accessibility of information: a randomized controlled trial.
      ], authors used multiple choice questions to asses participants' understanding of key messages of summary formats while reading summaries. Our aim was to assess the amount of information retained by participants after reading the review summary. For that reason, knowledge about the information from the review summary was assessed as a score on a test with 10 questions (maximum score = 10) (Additional File 1/Appendix at www.jclinepi.com). The questions were open ended and focused on learning and remembering the benefits and risks of the intervention and the quality of evidence described in the systematic review. The questions were developed by the authors and pilot tested for face validity with three independent experts.

      2.4.2 Secondary outcomes

      Reading experience and user-friendliness of the format were assessed by 5-question surveys, measured on a 10-point Likert type scale, ranging from 1—do not agree at all to 10—fully agree (maximum score = 50 for each scale) (Additional File 1/Appendix at www.jclinepi.com). We also collected the following information: gender (trials 1 and 3), age, knowledge about or prior experiences with Cochrane reviews and the Cochrane Library, as well as habits in searching for health information. The familiarity of the students with Cochrane summaries was assessed by the following questions in the survey: (1) “Have you ever heard of Cochrane systematic reviews before?”, (2) “Have you ever used the Cochrane library?”, and (3) “What is your primary source of health information?” Doctors were also asked whether they had a research (PhD) degree, if they were specialists and if they had an academic position. We also used the 6-item General Health Numeracy Test [
      • Osborn C.Y.
      • Wallston K.A.
      • Shpigel A.
      • Cavanaugh K.
      • Kripalani S.
      • Rothman R.L.
      Development and validation of the general health numeracy test (GHNT).
      ] to determine how much the participants understood basic health instructions regarding the numeracy dimension. The numeracy score was the sum of correct answers (maximum score = 6).

      2.5 Sample size

      The sample size was calculated using G*Power 3.1.9.2. [
      • Faul F.
      • Erdfelder E.
      • Lang A.G.
      • Buchner A.
      G*Power 3: a flexible statistical power analysis program for the social, behavioral, and biomedical sciences.
      ] software and was based on the primary outcome. We used an alpha of 0.05 and 80% power to detect a difference in the average knowledge score between groups of people who received the different formats. We used findings from previous similar studies [
      • Santesso N.
      • Rader T.
      • Nilsen E.S.
      • Glenton C.
      • Rosenbaum S.
      • Ciapponi A.
      • et al.
      A summary to communicate evidence from systematic reviews to the public improved understanding and accessibility of information: a randomized controlled trial.
      ,
      • Schwartz L.M.
      • Woloshin S.
      • Welch H.G.
      Using a drug facts box to communicate drug benefits and harms: two randomized trials.
      ] to define the effect size (f ≥ 0.5). We thus calculated that, at a minimum, 14 people in each intervention group were needed to complete the survey.

      2.6 Randomization

      2.6.1 Trial 1

      The schedule for the Medical Humanities course is organized in three seminar groups, to which students are normally assigned in alphabetical order. For the purposes of this study, we used an online randomization program (www.randomisation.com, permuted block method, seed No. 2197) to assign students into one of the three groups. This list was posted on the course website. Students from the School of Humanities and Social Sciences were randomized into three groups (seeds nos. 6549, 15121, 5020431), which then came to the School of Medicine for their scheduled class. Randomization using the online survey tool (SurveyMonkey) was not possible as it offers only two group randomization (https://help.surveymonkey.com/articles/en_US/kb/What-is-Random-Assignment).

      2.6.2 Trial 2

      The participants were randomized using the SurveyMonkey's A/B Test feature.

      2.6.3 Trial 3

      The doctors' emails were randomized into three groups using online software (www.randomization.com, permuted block method, seed no. 171). E-mail invitations were sent to three mailing lists, where each group received the link to a single summary format.

      2.7 Blinding

      The participants were blinded to the study design and randomization. The invitation for participation stated that the research addressed knowledge about health evidence and each participant was presented with only a single presentation format, without identifying information on its type. The investigators and data analysts (I.B., L.P.) were blinded to the group allocations. The same was true for the teachers in the student courses, who were independent from the research group.

      2.8 Data collection and analysis

      We excluded participants who did not complete the survey after starting it (Fig. 1). The data were collected in a fully anonymous manner and stored on a secure server at the School of Medicine. Answers to open-ended questions were rated by two independent raters (I.B. and L.P.), who demonstrated high degree of agreement in their assessments (kappa coefficient ranged from 0.9 to 1.0 for individual questions). Gender, awareness about Cochrane systematic reviews, prior use of the Cochrane Library, primary source of health information, specialization, PhD degree, and academic position were presented in percentages; median values and interquartile range were calculated for age and health numeracy scores; and median and 95% confidence interval (CI) for knowledge, reading experience, and user-friendliness scores. Reliability coefficients of scales measuring reading experience, user-friendliness, and health numeracy were calculated on aggregated data from all three trials and expressed as Cronbach's alpha. Groups were compared using Kruskal–Wallis nonparametric tests and post hoc Conover–Iman tests [
      • Conover W.J.
      Practical nonparametric statistics.
      ] in trials 1 and 3, and Mann–Whitney nonparametric tests in trial 2.
      Figure thumbnail gr1
      Fig. 1Flow of the participants in the study.
      We used logistic regression to determine significant predictors for the primary outcome. Participants were classified into “lower” or “higher” group in relation to the median value of each summary format group. Regression coefficient was expressed as Cox and Snell R2 [
      • Cox D.R.
      • Snell E.J.
      Analysis of binary data.
      ]. The covariates used as predictors were sample characteristics, including health numeracy, reading experience, and user-friendliness scores. The same predictors were used for aggregated data from all trials.
      The differences between groups across trials were tested using Kruskal–Wallis test and post hoc Conover–Iman test for infographic and PLS formats, and Mann–Whitney test for SA. We also performed t-tests across the trials to compare the effects of infographic vs. PLS on the measures of knowledge, reading experience, and user-friendliness. Effect sizes were expressed as Cohen's d [
      • Cohen J.
      Statistical power analysis for the behavioral sciencies.
      ].
      All statistical analyses were performed using the Statistical Package for Social Sciences (SPSS) 19 (IBM SPSS Statistics for Windows, version 19.0, Armonk, NY, USA) and Medcalc version 17.1 (Medcalc Software, Ostend, Belgium).

      3. Results

      The flow of participants for each of the three trials is presented in Fig. 1. The reliability of the scales used in the trials was Cronbach α = 0.77 for reading experience, α = 0.72 for user-friendliness, and α = 0.68 for health numeracy.

      3.1 Trial 1—students

      All students who attended the course participated in the trial (Fig. 1). The proportion of participants who did or did not complete the survey in the infographic and PLS groups was not statistically significant (χ2 = 1.87, df = 1, P = 0.171) in the consumer trial. There was also no difference between the doctors' groups in completion of the survey (χ2 = 4.32, df = 2, P = 0.115).
      The demographic characteristics of participants in the three format groups were similar (Table 1). The group that read the SA had the lowest median score on all outcomes (Table 2). The groups that read the infographic or PLS did not differ significantly in any of the measured outcomes (Table 2).
      Table 1Characteristics and health numeracy of participants in the three trials (N = 334)
      CharacteristicsInfographicPLSSA
      Trial 1: Students (N = 171)n = 54n = 60n = 57
       Female sex (n, %)44 (81.5)48 (80.0)50 (85.9)
       Heard of Cochrane systematic reviews (n, %)5 (9.4)2 (3.3)2 (3.5)
       Used the Cochrane Library (n, %)000
       Primary source of health information (n, %)
      Internet25 (46.3)31 (51.7)29 (50.8)
      Family members15 (27.8)11 (18.3)15 (26.3)
      Family physician11 (20.4)16 (26.7)11 (19.3)
      Library2 (3.7)2 (3.3)2 (3.7)
      Other1 (1.8)00
       Age (median, IQR)19 (19–21)19 (19–20)19 (19–21)
       Health numeracy (median, IQR)
      Results are expressed as a score on a test of six questions (maximum score 6).
      2.0 (1.0–3.0)2.0 (2.0–3.8)3.0 (2.0–4.0)
      Trial 2: Consumers (N = 99)n = 45n = 54
       Heard of Cochrane systematic reviews (n, %)25 (55.6)23 (42.6)
       Used the Cochrane Library (n, %)16 (35.6)12 (22.2)
       Primary source of health information (n, %)
      Internet33 (73.3)33 (61.1)
      Family members04 (7.4)
      Family physician5 (11.1)12 (22.2)
      Library1 (2.2)2 (3.7)
      Other6 (13.3)3 (5.6)
       Age (median, IQR)37.0 (32.0–41.8)39.0 (32.0–42.0)
       Health numeracy (median, IQR)
      Results are expressed as a score on a test of six questions (maximum score 6).
      5.0 (4.0–5.0)5.0 (4.0–6.0)
      Trial 3: Doctors (N = 64)n = 25n = 18n = 21
       Female sex (n, %)8 (32.0)9 (50.0)16 (76.2)
       Heard of Cochrane systematic reviews (n, %)23 (92.0)18 (100)20 (95.2)
       Used the Cochrane Library (n, %)17 (68.0)17 (94.4)15 (71.4)
       Specialization obtained (n, %)18 (72.0)13 (72.2)15 (71.4)
       PhD degree (n, %)13 (52.0)9 (50.0)7 (33.3)
       Research or teaching position (n, %)
      Neither research nor teaching position15 (60.0)9 (50.0)9 (42.9)
      Research position (research associate, higher scientific associate)5 (20.0)2 (11.1)2 (9.5)
      Academic (research-teaching) position5 (20.0)1 (5.6)4 (19.5)
      Other06 (33.3)6 (28.6)
       Age (median, IQR)36.0 (30.0–48.5)40.0 (29.0–54.5)38.0 (29.0–48.5)
       Health numeracy (median, IQR)
      Results are expressed as a score on a test of six questions (maximum score 6).
      5.0 (4.0–6.0)5.0 (5.0–6.0)5.0 (4.05.0)
      Abbreviations: PLS, plain language summary; SA, scientific abstract; IQR, interquartile range.
      a Results are expressed as a score on a test of six questions (maximum score 6).
      Table 2Participants' scores on knowledge about health information, reading experience, and user-friendliness of three summary information formats of a Cochrane systematic review
      Results are expressed as test scores for knowledge about health information (10 questions, maximum score 10) and for reading experience and user-friendliness scales (five questions each, with a Likert scale ranging from 0 to 10, maximum score 50).
      Presentation format (median, 95% CI)P
      Kruskal–Wallis test (students and doctors) and Mann–Whitney U test (consumers).
      InfographicPLSSA
      Trial 1: Students (N = 171)n = 54n = 60n = 57
       Knowledge6 (5.0–7.0)6 (5.1–6.0)5 (4.0–5.0)
      P < 0.05 vs. other two format groups, Conover–Iman post hoc test.
      <0.001
       Reading experience31 (27.8–33.0)30 (26.1–32.0)16 (14.3–18.0)
      P < 0.05 vs. other two format groups, Conover–Iman post hoc test.
      <0.001
       User-friendliness32.5 (30.0–34.5)32.5 (31.0–35.0)19.5 (17.0–21.0)
      P < 0.05 vs. other two format groups, Conover–Iman post hoc test.
      <0.001
      Trial 2: Consumers (N = 99)n = 45n = 54
       Knowledge7.0 (6.0–7.0)7.0 (6.0–7.0)0.511
       Reading experience33.0 (28.0–36.0)22.5 (19.0–27.4)
      P < 0.05 vs. doctors and consumers for PLS (Kruskal–Wallis test with Conover–Iman post hoc test).
      <0.001
       User-friendliness30.0 (25.5–34.5)21.0 (19.0–25.0)<0.001
      Trial 3: Doctors (N = 64)n = 25n = 18n = 21
       Knowledge8.0 (6.0–8.0)8.0 (7.0–9.0)
      P < 0.05 vs. students for SA (Mann–Whitney U test) or vs. students and consumers for two other formats (Kruskal–Wallis test with Conover–Iman post hoc test).
      8.0 (5.9–9.0)
      P < 0.05 vs. students for SA (Mann–Whitney U test) or vs. students and consumers for two other formats (Kruskal–Wallis test with Conover–Iman post hoc test).
      0.611
       Reading experience37.0 (26.8–41.3)32.0 (30.0–39.9)24.0 (21.3–31.8)
      P < 0.05 vs. students for SA (Mann–Whitney U test) or vs. students and consumers for two other formats (Kruskal–Wallis test with Conover–Iman post hoc test).
      0.002
       User-friendliness36.0 (30.9–40.0)29.0 (26.8–36.2)25.0 (23.5–27.2)
      P < 0.05 vs. students for SA (Mann–Whitney U test) or vs. students and consumers for two other formats (Kruskal–Wallis test with Conover–Iman post hoc test).
      0.003
      Abbreviations: PLS, plain language summary; SA, scientific abstract; CI, confidence interval.
      a Results are expressed as test scores for knowledge about health information (10 questions, maximum score 10) and for reading experience and user-friendliness scales (five questions each, with a Likert scale ranging from 0 to 10, maximum score 50).
      b Kruskal–Wallis test (students and doctors) and Mann–Whitney U test (consumers).
      c P < 0.05 vs. other two format groups, Conover–Iman post hoc test.
      d P < 0.05 vs. doctors and consumers for PLS (Kruskal–Wallis test with Conover–Iman post hoc test).
      e P < 0.05 vs. students for SA (Mann–Whitney U test) or vs. students and consumers for two other formats (Kruskal–Wallis test with Conover–Iman post hoc test).
      In the overall sample, the optimal combination of predictors of the primary outcome included health numeracy score [odds ratio (OR) = 1.72; 95% CI: 1.29–2.29] and reading experience (OR = 1.08; 95% CI: 1.04–1.13), explaining 18.3% of the variance.

      3.2 Trial 2—consumers

      A total of 212 participants started the survey, and 99 (47%) completed it (Fig. 1). The demographic characteristics of the participants were similar across the three format groups (Table 1). There were no significant differences between the infographic and PLS groups in the knowledge test score (Table 2). A significant predictor of the knowledge score was awareness about Cochrane systematic reviews (OR = 5.33; 95% CI: 1.71–16.62), explaining 13.4% of the variance.

      3.3 Trial 3—doctors

      Out of 270 e-mail invitations, a total of 108 doctors accessed the survey, and 64 (59%) completed it (Fig. 1).
      The demographic characteristics were similar across the three format groups, except there were more female participants in the SA group (Table 1). There were no significant differences among the three groups in the knowledge score (Table 2). However, the group that read the SA had significantly lower scores on the measures of reading experience and user-friendliness of the material (Table 2). None of the demographic characteristics or secondary outcomes significantly predicted the knowledge test score.

      3.4 Comparisons across trials

      We compared the group results on knowledge, reading experience, and user-friendliness across trials. In the PLS group, consumers reported lower satisfaction with that format, compared with students and doctors (Table 2). Doctors had significantly higher scores than students on the knowledge test, reading experience, and perceived user-friendliness in the SA group (Table 2). In the overall sample, the participants who received the infographic had a similar knowledge score to those reading the PLS (Table 3). However, infographic users scored higher on reading experience and user-friendliness than those reading the PLS (Table 3). Finally, we performed logistic regression on the overall sample from all three trials (N = 334) to determine which variables predicted the results on the knowledge test. Significant predictors were health numeracy (OR = 1.48; 95% CI: 1.25–1.74) and reading experience (OR = 1.06; 95% CI: 1.04–1.09), explaining 17.0% of the variance of the knowledge score.
      Table 3Knowledge about health information, reading experience, and user-friendliness of an infographic vs. plain language summary (PLS) of a Cochrane systematic review across trials
      Results are expressed as test scores for knowledge about health information (10 questions, maximum score 10) and for reading experience and user-friendliness scales (5 questions each, with a Likert scale ranging from 0 to 10, maximum score 50).
      Presentation format (mean, 95% CI)Mean difference (95% CI)P
      Student t-test for independent samples.
      Effect size (Cohen d)
      Infographic (n = 124)PLS (n = 132)
      Knowledge6.5 (6.2–6.9)6.3 (6.0–6.6)0.2 (−0.2 to 0.6)0.359
      Reading experience31.7 (30.1–33.3)27.3 (25.7–28.9)4.4 (2.1–6.7)<0.0010.48
      User-friendliness31.8 (30.3–33.3)27.9 (26.4–29.3)3.9 (1.7–6.1)<0.0010.45
      a Results are expressed as test scores for knowledge about health information (10 questions, maximum score 10) and for reading experience and user-friendliness scales (5 questions each, with a Likert scale ranging from 0 to 10, maximum score 50).
      b Student t-test for independent samples.

      4. Discussion

      In the three randomized trials that tested differences between text-based information and infographics, and included students, consumers, and doctors, we found no differences in readers' knowledge about the results of a Cochrane systematic review from the different formats.
      The results of the trials should be interpreted with the following limitations in mind. Each trial included small samples for each arm who read different formats, with the student trial including most participants. Additionally, we used only one specific systematic review with a clear finding (result), and therefore, our findings may not be representative for other (all) reviews, which may present less definite evidence. In the consumer and doctor groups, the rate of noncompletion of the survey was high, but there were no differences among the groups within the same trial. The participants who completed the survey presumably had a higher degree of motivation and may have been different from those who did not complete it. There were also differences in the way the groups took the survey, with consumers and clinicians taking the survey online, at their chosen time and with a shorter gap between reading the summary and testing comprehension than students who took the survey at medical school, in groups, and watched a film between reading and testing. The longer time frame between reading the information format and answering the survey for the student vs. consumer/doctor groups may have adversely affected their scores on the knowledge tests, holding only the basic informative knowledge which was processed through the working memory (Cowan, 2008). In addition, there is a possibility that participants answering the online survey (consumers and doctors) could have consulted other sources, as there was no time restriction for their survey response. In order to decrease the differences between groups in the time between reading the assigned summary format and taking the survey, the participants in the consumer and doctor groups were given a numeracy test between the reading of the information format and the knowledge test. The schedule of the numeracy test right after the information format could have also contributed to the high dropout rate for the two groups tested online. Finally, we did not test the learning preferences of the trial participants. Since we tested visual or textual material presented on computer screens, it may be different from the learning materials or information sources to which the participants were used.
      The finding that an infographic had similar scores as the PLS in terms of knowledge and remembering the key messages of a systematic review should be interpreted in view of the fact the PLS used in the trial was written by a professional writer and Cochrane reviewer to achieve a high level of readability and to conform with Cochrane PLEACS standards [
      • Cochrane
      Standards for the reporting of Plain Language Summaries in new Cochrane Intervention Reviews (PLEACS).
      ], whereas PLSs written by review authors often do not conform to the PLEACS standards [
      • Jelicic Kadic A.
      • Fidahic M.
      • Vujcic M.
      • Saric F.
      • Propadalo I.
      • Marelja I.
      • et al.
      Cochrane plain language summaries are highly heterogeneous with low adherence to the standards.
      ]. On the other hand, creating infographics requires significantly greater time resources and financial costs for graphic design than PLS writing. If our findings are generalizable to other situations, then the implication is that, to ensure readability and usability among lay audiences, the focus should be on producing high quality, easily readable PLSs.
      One of the strengths of this study was the use of open-ended questions for the knowledge outcome measure. Prior research on Cochrane PLSs [
      • Vandvik P.O.
      • Santesso N.
      • Akl E.A.
      • You J.
      • Mulla S.
      • Spencer F.A.
      • et al.
      Formatting modifications in GRADE evidence profiles improved guideline panelists comprehension and accessibility to information. A randomized trial.
      ] has used multiple choice questions with a single correct answer to assess understanding of PLS. However, multiple choice questions are closed questions, and the correct answer may provide a visual reminder to the participant and therefore may not measure real understanding or knowledge of the material [
      • Choi B.C.
      • Pak A.W.
      A catalog of biases in questionnaires.
      ]. In order to reduce this measurement bias, we used open-ended questions.
      The traditional SA gave the lowest scores when assessing knowledge, reading experience, and user-friendliness in the student trial, confirming that scientific formats and language of information presentation for systematic reviews are difficult for nonspecialist audiences. Despite the fact that the standard SA format is assumed to be the most appropriate tool for experts, who are looking for specific information about trials [
      • Crick K.
      • Hartling L.
      Preferences of knowledge users for two formats of summarizing results from systematic reviews: infographics and critical appraisals.
      ], doctors in our study preferred the other formats to the SA. We also found that consumers rated the infographic better for reading experience than the PLS. Although there was no difference in information translation between the two formats, participants preferred the infographic and rated it better for ease of use and overall reading experience [
      • Cohen J.
      Statistical power analysis for the behavioral sciencies.
      ], but with medium effect sizes and with only few points difference on a scale from 10 to 50 points. This suggests that an infographic presentation may be considered slightly more interesting or easier to read compared to PLS. If this is the case, optimizing the translation of information from systematic reviews to different audiences should focus on examining possible factors underlying reading experiences to develop formats most suitable for capturing reader's attention and make the learning process easier.
      The best predictor of the knowledge scores among consumers was the awareness about Cochrane systematic reviews. The finding from our study that participants who have heard of Cochrane reviews had higher knowledge scores may be due to the fact that they were more familiar with the information formats used in our study, providing support for Cochrane to continue to promote systematic review findings to the public to aid their interpretation of results from clinical research so they can participate fully in health choices. Moreover, the finding that significantly fewer participants reported having used the Cochrane Library in comparison to those who reported being aware of Cochrane reviews suggests that participants had heard about systematic reviews from their consumer group, rather than through searching for information themselves.
      Health numeracy was a significant predictor of knowledge in the student sample and in the overall sample. Although this finding may reflect the fact that there was a higher proportion of students in the aggregated data compared to other trial groups, numeracy is important in knowledge translation because low numeracy skills make understanding of health information more difficult [
      • Spiegelhalter D.
      • Pearson M.
      • Short I.
      Visualizing uncertainty about the future.
      ,
      • Greene J.
      • Peters E.
      Medicaid consumers and informed decisionmaking.
      ]. It has been shown that both patients and doctors have difficulties with interpretation of probabilistic findings [
      • Bramwell R.
      • West H.
      • Salmon P.
      Health professionals' and service users' interpretation of screening test results: experimental study.
      ]. One way to improve health numeracy in a professional population is greater emphasis on statistics and probability theory in the early years of medical school so that students apply evidence and critical thinking in medical practice [
      • Chalmers T.C.
      Dr. Tom Chalmers, 1917-1995: the trials of a randomizer. Interview by Malcom Maclure.
      ,
      • Marusic A.
      • Sambunjak D.
      • Jeroncic A.
      • Malicki M.
      • Marusic M.
      No health research without education for research–experience from an integrated course in undergraduate medical curriculum.
      ]. Moreover, a recent systematic review concluded that scientific findings are easier to interpret when results are presented as frequencies than probabilities [
      • Akl E.A.
      • Oxman A.D.
      • Herrin J.
      • Vist G.E.
      • Terrenato I.
      • Sperati F.
      • et al.
      Using alternative statistical formats for presenting risks and risk reductions.
      ]. These findings may be relevant for training efforts for doctors, who have been shown to have difficulties in interpreting the results of systematic reviews [
      • Lai N.M.
      • Teng C.L.
      • Lee M.L.
      Interpreting systematic reviews: are we ready to make our own conclusions? A cross-sectional study.
      ]. Furthermore, experts preparing abstracts and plain language summaries should consider that consumers generally have low health numeracy [
      • Sorensen K.
      • Pelikan J.M.
      • Rothlin F.
      • Ganahl K.
      • Slonska Z.
      • Doyle G.
      • et al.
      Health literacy in Europe: comparative results of the European health literacy survey (HLS-EU).
      ], and they should therefore aim to present information in such a way that nonspecialists are able to understand the key messages.

      5. Conclusion

      We found that PLS and infographic formats were equally effective in terms of transmitting knowledge about the findings from a systematic review. However, the infographic format may be perceived as more enjoyable to read than either the PLS format or a standard scientific abstract.

      References

        • Kurtzman E.T.
        • Greene J.
        Effective presentation of health care performance information for consumer decision making: a systematic review.
        Patient Educ Couns. 2016; 99: 36-43
        • Maguire L.K.
        • Clarke M.
        How much do you need: a randomised experiment of whether readers can understand the key messages from summaries of Cochrane Reviews without reading the full review.
        J R Soc Med. 2014; 107: 444-449
        • Alderdice F.
        • McNeill J.
        • Lasserson T.
        • Beller E.
        • Carroll M.
        • Hundley V.
        • et al.
        Do Cochrane summaries help student midwives understand the findings of Cochrane systematic reviews: the BRIEF randomised trial.
        Syst Rev. 2016; 5: 40
        • Gagliardi A.R.
        • Legare F.
        • Brouwers M.C.
        • Webster F.
        • Badley E.
        • Straus S.
        Patient-mediated knowledge translation (PKT) interventions for clinical encounters: a systematic review.
        Implement Sci. 2016; 11: 26
        • Langendam M.W.
        • Akl E.A.
        • Dahm P.
        • Glasziou P.
        • Guyatt G.
        • Schunemann H.J.
        Assessing and presenting summaries of evidence in Cochrane Reviews.
        Syst Rev. 2013; 2: 81
        • Santesso N.
        • Rader T.
        • Nilsen E.S.
        • Glenton C.
        • Rosenbaum S.
        • Ciapponi A.
        • et al.
        A summary to communicate evidence from systematic reviews to the public improved understanding and accessibility of information: a randomized controlled trial.
        J Clin Epidemiol. 2015; 68: 182-190
        • Jelicic Kadic A.
        • Fidahic M.
        • Vujcic M.
        • Saric F.
        • Propadalo I.
        • Marelja I.
        • et al.
        Cochrane plain language summaries are highly heterogeneous with low adherence to the standards.
        BMC Med Res Methodol. 2016; 16: 61
        • Comello M.L.
        • Qian X.
        • Deal A.M.
        • Ribisl K.M.
        • Linnan L.A.
        • Tate D.F.
        Impact of game-inspired infographics on user engagement and information processing in an eHealth program.
        J Med Internet Res. 2016; 18: e237
        • Crick K.
        • Hartling L.
        Preferences of knowledge users for two formats of summarizing results from systematic reviews: infographics and critical appraisals.
        PLoS One. 2015; 10: e0140029
        • Arcia A.
        • Suero-Tejeda N.
        • Bales M.E.
        • Merrill J.A.
        • Yoon S.
        • Woollen J.
        • et al.
        Sometimes more is more: iterative participatory design of infographics for engagement of community members with varying levels of health literacy.
        J Am Med Inform Assoc. 2016; 23: 174-183
        • Hofmeyr G.J.
        • Kulier R.
        • West H.M.
        External cephalic version for breech presentation at term.
        Cochrane Database Syst Rev. 2015; : CD000083
        • Cochrane
        Standards for the reporting of Plain Language Summaries in new Cochrane Intervention Reviews (PLEACS).
        ([cited 2016 26 November]; Available at) (Accessed November 1, 2017)
        • Osborn C.Y.
        • Wallston K.A.
        • Shpigel A.
        • Cavanaugh K.
        • Kripalani S.
        • Rothman R.L.
        Development and validation of the general health numeracy test (GHNT).
        Patient Educ Couns. 2013; 91: 350-356
        • Bell D.S.
        • Harless C.E.
        • Higa J.K.
        • Bjork E.L.
        • Bjork R.A.
        • Bazargan M.
        • et al.
        Knowledge retention after an online tutorial: a randomized educational experiment among resident physicians.
        J Gen Intern Med. 2008; 23: 1164-1171
        • Faul F.
        • Erdfelder E.
        • Lang A.G.
        • Buchner A.
        G*Power 3: a flexible statistical power analysis program for the social, behavioral, and biomedical sciences.
        Behav Res Methods. 2007; 39: 175-191
        • Schwartz L.M.
        • Woloshin S.
        • Welch H.G.
        Using a drug facts box to communicate drug benefits and harms: two randomized trials.
        Ann Intern Med. 2009; 150: 516-527
        • Conover W.J.
        Practical nonparametric statistics.
        3rd ed. John Wiley & Sons, New York1999
        • Cox D.R.
        • Snell E.J.
        Analysis of binary data.
        2nd ed. Chapman & Hall, London1989
        • Cohen J.
        Statistical power analysis for the behavioral sciencies.
        Routledge, Abingdon, 1977
        • Vandvik P.O.
        • Santesso N.
        • Akl E.A.
        • You J.
        • Mulla S.
        • Spencer F.A.
        • et al.
        Formatting modifications in GRADE evidence profiles improved guideline panelists comprehension and accessibility to information. A randomized trial.
        J Clin Epidemiol. 2012; 65: 748-755
        • Choi B.C.
        • Pak A.W.
        A catalog of biases in questionnaires.
        Prev Chronic Dis. 2005; 2: A13
        • Spiegelhalter D.
        • Pearson M.
        • Short I.
        Visualizing uncertainty about the future.
        Science. 2011; 333: 1393-1400
        • Greene J.
        • Peters E.
        Medicaid consumers and informed decisionmaking.
        Health Care Financ Rev. 2009; 30: 25-40
        • Bramwell R.
        • West H.
        • Salmon P.
        Health professionals' and service users' interpretation of screening test results: experimental study.
        BMJ. 2006; 333: 284
        • Chalmers T.C.
        Dr. Tom Chalmers, 1917-1995: the trials of a randomizer. Interview by Malcom Maclure.
        CMAJ. 1996; 155: 757-760
        • Marusic A.
        • Sambunjak D.
        • Jeroncic A.
        • Malicki M.
        • Marusic M.
        No health research without education for research–experience from an integrated course in undergraduate medical curriculum.
        Med Teach. 2013; 35: 609
        • Akl E.A.
        • Oxman A.D.
        • Herrin J.
        • Vist G.E.
        • Terrenato I.
        • Sperati F.
        • et al.
        Using alternative statistical formats for presenting risks and risk reductions.
        Cochrane Database Syst Rev. 2011; : CD006776
        • Lai N.M.
        • Teng C.L.
        • Lee M.L.
        Interpreting systematic reviews: are we ready to make our own conclusions? A cross-sectional study.
        BMC Med. 2011; 9: 30
        • Sorensen K.
        • Pelikan J.M.
        • Rothlin F.
        • Ganahl K.
        • Slonska Z.
        • Doyle G.
        • et al.
        Health literacy in Europe: comparative results of the European health literacy survey (HLS-EU).
        Eur J Public Health. 2015; 25: 1053-1058

      Linked Article