If you don't remember your password, you can reset it by entering your email address and clicking the Reset Password button. You will then receive an email that contains a secure link for resetting your password
If the address matches a valid account an email will be sent to __email__ with instructions for resetting your password
Evidence-Based Research Series-Paper 3: Using an Evidence-Based Research approach to place your results into context after the study is performed to ensure usefulness of the conclusion
Faculty of Health, Institute of Sport Science and Clinical Biomechanics, University of Southern Denmark, Odense, DenmarkDepartment of Physiotherapy and Occupational Therapy, University Hospital of Copenhagen, Herlev & Gentofte, Copenhagen, Denmark
Department of Health, Public Health and Epidemiology Group, Science and Technology, Aalborg University, Alborg, DenmarkDepartment of Physical and Occupational Therapy, Aalborg University Hospital, Aalborg, Denmark
Musculoskeletal Statistics Unit, The Parker Institute, Bispebjerg and Frederiksberg Hospital, Copenhagen, DenmarkDepartment of Clinical Research, Research Unit of Rheumatology, University of Southern Denmark, Odense University Hospital, Denmark
Section for Evidence-based Practice, Western Norway University of Applied Sciences, Bergen, NorwaySchool of Nursing, McMaster University, Hamilton, Ontario, Canada
MEDICI Centre, Schulich School of Medicine & Dentistry, Western University, London, Ontario, CanadaDepartments of Anesthesia & Perioperative Medicine and Epidemiology & Biostatistics, Western University, London, Ontario, Canada
Johns Hopkins Evidence-based Practice Center, Division of General Internal Medicine, Department of Medicine, Johns Hopkins University, Baltimore, MD, USA
There is considerable actual and potential waste in research. Using evidence-based research (EBR) can ensure the value of a new study. The aim of this article, the third in a series, is to describe an EBR approach to putting research results into context.
Study Design and Setting
EBR is the use of prior research in a systematic and transparent way to inform a new study so that it is answering questions that matter in a valid, efficient, and accessible manner. In this third and final article of a series, we describe how to use the context of existing evidence to reach and present a trustworthy and useful conclusion when reporting results from a new clinical study.
Results
We describe a method, the EBR approach, that by using a systematic and transparent consideration of earlier similar studies when interpreting and presenting results from a new original study will ensure usefulness of the conclusion.
Conclusion
Using an EBR approach will improve the usefulness of a clinical study by providing the context to draw more valid conclusions and explicit information about new research needs.
The conclusion of a new study should not be based on the results from the new study alone but by a synthesis of existing evidence and new results. The interpretation and reporting of results from a new study should be within the context of what is already known.
What this adds to what was known?
•
This article describes the evidence-based research approach to making and reporting conclusions. This approach includes an update of the systematic review that was used to justify and design the study and a process to decide if there are implications for clinical practice or if further research is needed.
What is the implication and what should change now?
•
To ensure valid and valuable studies, researchers should adopt the evidence-based research approach to determine and report the implications of the new study results or practice and for future research by explicitly considering the existing evidence.
1. Introduction
Evidence-Based Research (EBR) is the use of prior research in a systematic and transparent way to inform a new study so that it is answering questions that matter in a valid, efficient, and accessible manner [
]. In the previous article in our EBR series, we discussed the cumulative nature or science and showed the importance of justifying and designing new studies based on existing knowledge (see article #2) [
]. In this third article of the series, we focus on the use of an EBR approach after the completion of a study and make recommendations on how to interpret and report the results of a new study in the context of the existing evidence base. We argue that placing the new study results in the context of what is already known is a key requisite to creating a meaningful publication that lets readers assess the value added by the new study, its internal and external validity, and any similarities and differences between available studies to support decision-making in clinical practice and for future research [
The importance of these characteristics of good science has been clearly stated in the reporting standards for clinical studies. The CONSORT statement, first published in 1996, noted the need to “State general interpretation of the data in light of the totality of the available evidence” [
]. When updated in 2001, the CONSORT statement included more detailed recommendations on how to place new results in the context of previous studies, not only recommending comparing new results with other published studies, but that whenever possible, this should be performed by using a systematic review [
]. This recommendation was re-emphasized in 2010: “Readers will want to know how the present trial's results relate to those of other RCTs. This can best be achieved by including a formal systematic review in the results or discussion section of the report” [
A single study can very rarely (if ever) provide a definitive answer to the question investigated. Therefore, placing the new study in the context of relevant previous research is key, and metaresearch has shown that the interpretation of new results is at high risk of being biased if only a subset of earlier studies is included in the discussion of these new results [
]. Clarke et al. repeated investigations of whether clinical trials, published between 1997 and 2012 in five high-impact medical journals (Annals of Internal Medicine, BMJ, JAMA, The Lancet, and the New England Journal of Medicine), interpreted the new trial results by presenting an updated systematic review [
]. In 2009, 13 years after the publication of the first edition of the CONSORT statement, they concluded that only one study out of 29 examined contained an updated systematic review integrating the new results [
] and in subsequent updates found no evidence of progress in reporting results using systematic reviews from 1997 to 2012.
1.2 Suggested solution: the evidence-based research approach
The best way to place new results in the context of existing evidence is to include the new results in a systematic review. In this way, the investigator of the new study avoids the risk of a biased selection of earlier studies and ensures a trustworthy synthesis of earlier studies.
Preparing a systematic review from scratch is time intensive and effort intensive [
]. Identifying or preparing a systematic review during the planning phase of the new study will considerably lessen the required work when aiming to place results in context. Consequently, the suggested EBR approach described in this third article should be considered alongside the processes for how to justify the need for a new study and how to design a new study based on the totality of earlier similar studies (REF to EBR series article #2).
The relevant steps for this phase of the EBR approach are highlighted in Figure 1 and described in more detail in the next section.
Fig. 1The evidence-based research approach, here highlighting the approach after completion of the study.
], we propose the following EBR process to place the results from a new study in the context of prior studies.
Considering the impact of the new study results on the existing evidence raises similar issues as during the planning phase of the study when deciding whether to update a systematic review [
]. The relevant process is illustrated in Figure 1, article 2 in this series, where it is also described how investigators should identify an updated version of the existing systematic review (and if that is not available, update the systematic review) and how they should use it to justify and design their new study (REF til article #2 in this series). The results of the up-to-date systematic review can then be used as the context for interpreting and discussing the new findings [
]. If the systematic review used during the planning phase of the study contained a meta-analysis and the necessary data from earlier studies are available, updating the meta-analysis with the new results should be straightforward. At the completion of the new study, the researcher should assess the effect on the magnitude and precision of the effect when adding the new study to the pre-existing systematic review and assess whether and how the new results affect the conclusion and level of certainty.
Ellis et al. provide an example of adding the results of the new study to an existing meta-analysis in their investigation of taxane as adjuvant chemotherapy for patients with early breast cancer [
]. When embarking on the study in 2000, only a few studies had presented initial results, but by the time they were preparing to report the study results in 2009, a simple search using the search terms ‘taxane’ and ‘breast cancer’ identified a systematic review in the Cochrane Library performed by Ferguson et al. [
]. Ellis et al. were able to update these meta-analyses by adding the results of their new study. Thus, when discussing the impact for future clinical practice and research, the key findings included not just the results from the new study alone, but a much more meaningful estimate and conclusion based on the combined results of all studies examining the same clinical question.
Merry et al. did not conduct a meta-analysis including the results of their study on computerized self-help intervention for adolescents seeking help for depression (abbreviated SPARX) but discussed their own results within context of all the earlier studies [
The effectiveness of SPARX, a computerised self help intervention for adolescents seeking help for depression: randomised controlled non-inferiority trial.
Computerised cognitive behavioural therapy for the prevention and treatment of depression and anxiety in children and adolescents: a systematic review.
]. In addition, as data were sparse on adolescents, they complemented this by discussing another systematic review that included studies on adults and showed a similar positive effect to that found in earlier studies and their own new study with adolescents [
]. This way, the authors were able to draw confident conclusions about the benefit of the self-help intervention tested.
Once the new findings are combined with existing evidence, be it through a meta-analysis or not, the next question is as follows: Is it possible to draw a definitive conclusion or is further research needed? If a definitive conclusion can be drawn, this leads to a second question: Do we have confidence in this conclusion, or in other words, is the evidence of sufficiently high certainty? This process is very similar to the one described in article #2 of this series, where we established the need to take the ethical dimension and the grading of the evidence (including the statistics) into consideration when assessing the quality of the evidence (REF to article #2). Again, if the answer to this question is no, further research is needed. However, if our confidence in the conclusion is high, no further studies are needed, and a recommendation for or against the use of the intervention in clinical practice can be made (see Figure 2.).
Fig. 2The process steps from combining the results from the new study with existing evidence to the decision whether new research is needed, or if a recommendation for or against the use of the intervention in clinical practice can be made.
Finally, authors should formulate implications for clinical practice and suggestions for future research based on the totality of the evidence, highlighting the contribution of the new study. We recognize that numerous factors must be considered when evaluating the implications of a new study for both clinical practice and research but want to stress the role of the EBR approach as a vital step in this process.
1.4 Discussion
In this final article of our three-part series, we discussed how to use the EBR approach to reach and present a trustworthy and meaningful conclusion when reporting the results of a new clinical study. If new findings are not placed in the context of all earlier similar studies, the conclusion is at high risk of being biased. As a result, interventions without real effects may be introduced into clinical care or there may be erroneous recommendations that further studies are needed leading to new redundant studies and so increased research waste.
The focus for evidence-based clinical decisions needs to move beyond the point estimate and confidence interval of the single study to considering the aggregate estimate (and confidence and prediction intervals) from the accumulated evidence. If the certainty of the overall evidence is either unclear or low, more research is needed, and reporting this fact will help mitigate future risk of unnecessary medical reversal (premature uptake of therapies into practice) [
]. With a high certainty of the evidence, clinicians can use the results for clinical decision-making and researchers will be able to avoid repeating conclusive clinical research, thus avoiding wasteful redundant studies. With its focus on the totality of evidence and end users’ perspective, EBR is directly linked to two of the three key components of evidence-based medicine, an objective assessment of all relevant evidence, combined with patient values and circumstances, when making decisions for clinical care [
Researchers implementing the EBR approach will require more explicit guidance about how to interpret and report study results to ensure that study reports effectively support clinical decision-making and future research.
Acknowledgments
This work has been prepared as part of the Evidence-Based Research Network (ebrnetwork.org). The EBRNetwork is an international network that promotes the use of systematic reviews when prioritizing, designing, and interpreting research. Evidence-based research is the use of prior research in a systematic and transparent way to inform the new study so that it is answering questions that matter in a valid, efficient, and accessible manner.
The authors thank the Centre for Evidence-Based Practice, Western Norway University of Applied Sciences for their very generous support of the EBRNetwork.
The Parker Institute, Bispebjerg and Frederiksberg Hospital (Professor Christensen and Professor Henriksen) are supported by a core grant from the Oak Foundation USA (OCAY-18-774-OFIL).
Financial support
This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.
The effectiveness of SPARX, a computerised self help intervention for adolescents seeking help for depression: randomised controlled non-inferiority trial.
Computerised cognitive behavioural therapy for the prevention and treatment of depression and anxiety in children and adolescents: a systematic review.
31096-9&id=gr1.jpg){kind=link}
31096-9&id=gr2.jpg){kind=link}