Evidence-Based Research Series-Paper 2 : Using an Evidence-Based Research approach before a new study is conducted to ensure value

Background and Objectives: There is considerable actual and potential waste in research. The aim of this article is to describe how using an evidence-based research approach before conducting a study helps to ensure that the new study truly adds value. Study Design and Setting: Evidence-based research is the use of prior research in a systematic and transparent way to inform a new study so that it is answering questions that matter in a valid, efﬁcient, and accessible manner. In this second article of the evidence-based research series, we describe how to apply an evidence-based research approach before starting a new study. Results: Before a new study is performed, researchers need to provide a solid justiﬁcation for it using the available scientiﬁc knowledge as well as the perspectives of end users. The key method for both is to conduct a systematic review of earlier relevant studies. Conclusion: Describing the ideal process illuminates the challenges and opportunities offered through the suggested evidence-based research approach. A systematic and transparent approach is needed to provide justiﬁcation for and to optimally design a relevant and necessary new study. (cid:1) 2020 The Authors. Published by Elsevier Inc. This is an open access article under the CC BY license (http:// creativecommons.org/licenses/by/4.0/).


Introduction
This article is part of a series describing evidence-based researchdthe use of prior research in a systematic and transparent way to inform a new study so that it is answering questions that matter in a valid, efficient, and What is new?

Key findings
An evidence-based research approachdthe use of existing evidence in a transparent and explicit waydis needed to justify the need for and design a new study.

What this adds to what was known?
Researchers are given guidance on why and how to use an evidence-based research approach to justify and design a new study.

What are the implications and what should change now?
To ensure that only valuable studies are conducted in future, researchers should adopt the evidencebased research approach for justifying and designing a new study.
accessible manner [1].By prior research, we mean original studies (also called primary studies), but even when planning a new systematic review (secondary study), the authors should perform a comprehensive search for earlier similar systematic reviews to avoid redundancy [2,3].In this second of a three-part series, we describe how using the evidence-based research approach before starting a study helps to ensure its value.
Funding agencies and research ethics committees are key gatekeepers of the scientific process by reviewing new study protocols to evaluate their validity.If the design and chosen methods align with and seem appropriate to answer the proposed research question, the study is judged to be valid.In addition, if the recruitment of and all dealings with human participants (and their data) is ethically acceptable, the new study is usually approved and supported.However, a further ethical dimension needs to come into focus before the new study is allowed to go ahead: Is the proposed study worthwhile?Does it add true value?
The extreme circumstances of formulating an indictment against the investigated physicians in the Nuremberg processes (1946e47) after the Second World War led to the formulation of an ethical code that included the following statement [4]: ''2.The experiment should be such as to yield fruitful results for the good of society, unprocurable by any other methods or means of study, and not random or unnecessary in nature.''As Benjamin Freedman states, ''These principles seem to require as ethical preconditions that the study be of some value, and not simply be valid.''[4].
How can a new study be demonstrated to add value?Emanuel et al. provided an answer by reversing the argument, stating that research that is not ''socially or scientifically valuable includes clinical research with nongeneralizable results, a trifling hypothesis, or substantial or total overlap with proven results.''[5].Hence, if the relevant question raised by the new study has already been answered elsewhere, or where a substantial or total overlap exists with the available evidence base, the new study is unnecessary and of limited value.
A systematic review of earlier studies that addresses questions similar to the one to be investigated by the new study can identify evidence gaps and the presence of an 'overlap with proven results' [5].This will help ensure that any new study is necessary.However, judgments about value should include not only the identification of an evidence gap but also the perspectives of the end users, ensuring that the new study is of relevance to those it affects and to society.If an identified evidence gap cannot be shown to be relevant (based on the perspectives of the end users), it may not need to be filled.In addition, similarly, a need identified by end users does not in itself constitute a gap in our knowledge [6].Today a number of tools and approaches exist that help include end users' perspectives (see e.g., http://www.jla.nihr.ac.uk/; https://www.involve.org.uk/; or https://www.patientslikeme.com/).
Other factors such as the availability of funding, access to relevant technologies, and the competency of involved researchers will also be strong determinants whether a new study should be conducted, but an evidence-based research approach should always be applied first.

The operationalization of the evidence-based research approach
To assess whether the new study would indeed be filling an evidence gap, the existing evidence should be identified and synthesized systematically and transparentlyd currently this is performed by conducting a systematic review of earlier studies that answer the same research question.
The use of an evidence-based research approach during the planning phase of a study can be illustrated as shown in Figure 1.Whenever health researchers are planning a new study, a systematic review of earlier similar studies should be identified (and if need be updated) or conducted.In addition, a systematic and transparent gathering of the relevant end users' perspectives should be undertaken.
If the answer is no to the question of value of the new study, the figure shows that the researchers must consider a new angle to the suggested research question or identify another research question altogether, and then once more consider the value of this new question.Only if the researchers can clearly demonstrate that the intended study will add value, that is, there is both a societal need for it and it will fill a shown evidence gap should a new study be planned and designed in more detail.

Identifying gaps in the present knowledge
Any identification of an evidence gap related to a specific clinical question must be based on a systematic review of earlier similar studies, to acquire an unbiased and trustworthy knowledge of the existing evidence, to avoid unnecessarily repeating a study for which the answer is already known [7].Previous studies also provide researchers the opportunity to reflect on and determine the optimal study designda duty they owe the patients participating in their new study (see section 5 in the following).
Unfortunately, numerous metaresearch studies show that researchers continue to perform unnecessarily redundant research, when results from similar studies provide adequate evidence to address the question [8e17].It is very rare that researchers use systematic reviews to justify or design a new study [18e24].Typically, they refer only to a very small proportion of the original similar studies [10,20,25e28], and there does not seem to be a relationship between the number of earlier studies available to refer to and the number of studies the researchers actually cite.The reasons for selecting references for a study seem to be subjective [29e31] and based on preferences and strategic considerations rather than on a systematic and transparent approach.Positive, significant, and supportive studies are much more commonly cited than those that are negative, nonsignificant, or critical [28,32e37].

Identifying an existing or preparing a new systematic review when planning a new study
Conducting a systematic review from scratch demands specialized knowledge, skills, and a considerable amount of time [38,39] as well as additional expertise from subject matter experts such as specialized librarians and statisticians.An immense effort is put worldwide into preparing systematic reviews and developing the underlying methodology.One great example here is Cochrane, which has laid the foundations for gold standard systematic reviewing since its launch in 1993 and continues to develop and enhance the methods for the updating and preparation of systematic reviews.A more recent development is the International Collaboration for the Automation of Systematic Reviews (ICASR, https://icasr.github.io/)[40] that is stimulating and supporting technical initiatives to increase the efficiency and speed of the systematic review process.Most importantly, the number of globally published systematic reviews is growing rapidly [41], meaning that in future, health researchers will have a higher chance to identify and use existing systematic reviews for justifying and designing their new study.Unfortunately, among these is also an increasing number of irrelevant and redundant reviews, making quality appraisal ever more important [2,3].
Early on during the planning of a new study, health researchers should establish whether a relevant systematic review exists.Table 1 lists a number of recommended sites to search for existing systematic reviews.
If a published systematic review evaluating the same research question is identified, the next step is to determine whether it is of sufficient quality by using tools such as risk of bias in systematic reviews (ROBIS) [42] or AMSTAR-2 [43].
Once we know that we can have confidence in the results of the systematic review, the researcher needs to evaluate whether the systematic review is up to date or not.A generally applicable threshold for currency of reviews does not exist, but a review based on a search within a year will, in most cases, be current, whereas a review based on a search older than 5 years will probably not be current.Specifying a precise threshold would be misleading; the decision must be based on an assessment using specialist knowledge of the clinical field in which the research study will take place and the speed at which its evidence base evolves.If the systematic review is found to be out of date, the researcher needs to initiate updating (either in-house or outsourced) to identify, appraise, and incorporate more recent studies to base the planning of the new study on a comprehensive and up-to-date knowledge of all preexisting evidence.
Once all three components have been evaluated, so it is determined that the systematic review is relevant for the research question, up to date and of acceptable qualityd and its conclusions call for further researchdthe researcher can proceed to use it to justify their planned study and to inform its design.
However, if no relevant and adequate systematic review is found, the researcher needs to either prepare such a systematic review themselves or outsource the task to experts for preparing systematic reviews.If the identified systematic review is of acceptable quality but outdated, the researcher will need to update the search for original studies.This is necessary to ensure that a study published after the search date of the systematic review does not negate the need for a new study.

2.2.
Interpreting the results from the identified systematic review: is there an evidence gap?
When deciding if an evidence gap exists or not based on the identified systematic review, health researchers should consider three elements: the ethical element, the quality grading of the evidence found in the systematic review, and the use of statistical methods to support the decision process.

The ethical element
In 2000, Emanuel, Wendler, and Grady proposed seven requirements for ethical clinical research studies [5].In addition to respect for the participants, informed consent, independent review, a favorable risk ratio, fair subject selection, and scientific validity, they also ask health researchers to consider if the proposed research will enhance knowledge or health, and therefore have value.
Their recommendations are formulated as two questions to consider in determining if a new study is ethical [5]: (a) Is the research question scientifically valid and not a trifling hypothesis?(b) Will it be possible to generalize the new results, that is, beyond the sample and context of the study?
2.2.2.Grading the quality of the existing evidence When a systematic review is found, updated, or prepared, the certainty of the evidence for the existing body of evidence should be determined [44].Several different grading systems have been developed (see e.g., [44,45]).If confidence in the conclusion is high, that is, if the quality grading of the evidence indicates that the certainty of the evidence is high, there is no need for a new study, but if confidence is low or the evidence is insufficient, there is a case to be made for a new study.

Additional statistical methods
The interest in deciding when a meta-analysis is conclusive has led to the development of different statistical methods.Besides the obvious use of the confidence interval, other methods have been suggested such as the prediction interval (Section 10.10.4.3 in [46]), funnel plots [47], trial sequential analysis [48], and others (see examples in [49]).The prediction interval, for example, includes the heterogeneity of the included studies when calculating the probable range within which the true effect lies (10.10.4.3 in [46]).One could argue that in cases with a broader prediction interval compared with the confidence interval, the underlying higher heterogeneity necessitates further studies, that is, no further studies needed if the lower limit of the prediction interval is higher than the minimal clinical important difference threshold.
To sum up one can be fairly confident that a new study needs to be carried out if the answers to the two ethical questions are ''yes'', if the grading of the evidence for the conclusion from the systematic review indicates that the certainty of evidence is low or very low, and if statistical methods support low certainty of evidence.The final decision whether to conduct a study will come down to a nuanced consideration of the balance between the existing evidence, the relevance of the topic, and the opportunity cost of the research.

What if there are no earlier studies
During the early days of a new treatment or diagnostic method, it can be difficult to identify any earlier similar studies for the research in question.Even if the intervention is genuinely new, a systematic review should be conducted.It should be easy and fast to perform and appraise the resulting search to conclusively document that no earlier studies exist.In such cases, the main focus will be to make sure that the intervention is relevant for end users and society (see Section 3 in the following).
As stated in the explanation for CONSORT guidelines, it would also be important to consider if any ''plausible explanation for how the intervention under investigation might work, especially if there is little or no previous experience with the intervention'' [50].The example in Box 1 illustrates a possible approach in cases when no earlier similar original studies can be found.

Box 1
Researchers are planning a randomized controlled trial (RCT) for a recently developed physiotherapy treatment for patients with balance deficits, for example, due to sclerosis or Parkinsonism and want to use an evidence-based research approach to justify their study.They decide to perform a qualitative study to obtain the perspectives of patients and clinicians who will either receive or prescribe this new intervention (See Section 3 in the following) and a scoping review to understand the present evidence for other nonpharmacological treatments for patients with balance deficits.

Establishing whether a new study is relevant
In addition to the identification of evidence gaps through the use of systematic review, the need for a new study should be justified by establishing that it is addressing a question relevant to its end users.In the health context, end users usually encompass patients, caregivers, and clinicians.Patients and their caregivers may often be represented by national patient organizations (e.g., the American Heart Organization or the Norwegian Rheumatic Association).In other circumstances, carefully selected individuals, committees, or boards arranged by researchers may be used to elicit patient perspectives.Likewise, clinicians also have their professional organizations or can provide input as individuals invited to a panel or a committee.
The US Patient-Centered Outcomes Research Institute defines patient-relevant research as ''the evaluation of questions and outcomes meaningful and important to patients and caregivers.The definition rests on the axiom that patients have unique perspectives that can change and improve the pursuit of clinical questions.including the perspectives of end users of the research, which include patients, physicians, and other health care stakeholders, will enhance the relevance of research to actual health decisions these end users face'' [51].Within the last 20 years, and accentuated within the last 5e10 years, funders, regulators, and others supporting health research have been demanding the involvement of end users in research.
Patients have first-hand lived experience providing insight as to the most important matters in relation to a study of a specific patient group.Numerous studies show a discrepancy between what end users need and what researchers focus on, indicating that researchers are very poor at identifying or ignore the needs of end users when planning new research [52e59].In addition, public funding of research only modestly correlates with disease burden, if at all [60e62].

How to include the end user perspective?
The evidence-based research approach does not prescribe any particular methodology for involving end users in the research process but suggests that whatever is chosen should be carried out systematically and transparently (see Box 2).The optimal approach would be to conduct a systematic review of qualitative studies or surveys identifying experiences and attitudes among patients or clinicians about the disease and/or treatment, or diagnostic technique in question.
The challenge of using systematic reviews of qualitative studies or surveys is that the systematic review will most probably not cover all necessary aspects of the project.Thus, many health researchers have chosen to obtain end users' perspectives by inviting them to be a member of a panel, committee, board, or even of the research group itself [63].Unfortunately, this approach increases the risk of a nonsystematic and nontransparent involvement of end users.We therefore suggest to look for systematic reviews of qualitative studies or surveys obtaining the perspectives of the relevant end users, even though currently the number of such systematic reviews is sparse [64] compared with that of systematic reviews of randomized clinical trials [41].

The interaction between identifying evidence gaps and including end users' perspectives
As illustrated in Figure 1, both the end user's perspectives and results of a relevant systematic review of similar earlier studies need to be considered when aiming to establish the need for a new study.Moreover, the two elements interact with each other.If end users are asked to identify and prioritize the most relevant research questions, it would only be fair to inform them about the existing evidence base to avoid wasting time discussing research questions that have already been answered.Methods such as the evidence gap maps, systematic maps, evidence maps, evidence mapping and alike (see [65]) have been successfully used for this purpose.
In cases where there are no earlier studies (like the case with the new nonpharmacological treatment for balance deficits among neurological patients, see Box 1), end users' perspectives may well be the sole source for answering the question regarding the value of a new study.

Applying evidence-based research principles to the design of a new study
Study design is determined by a range of methodological options, such as the ways to collect and assign samples, to collect and analyze data, and to use interventions and instruments.Applying an evidence-based research approach will closely link the research question with the chosen Box 2 A new rehabilitation program for patients surviving a cardiac arrest was to be developed and tested.The PhD student applied an evidence-based research approach to planning and, instead of straightaway designing a new randomized clinical trial, first prepared the following three studies: The first study was a national population-based survey aiming to describe the self-reported prevalence of cognitive, psychological, and physical problems in people surviving a cardiac arrest and to analyze how these change over time.The study provided information about end users' perspectives in a systematic and transparent way.This could also have been achieved by conducting a systematic review of earlier published qualitative studies that had asked cardiac arrest survivors what was important to them when it came to their need for rehabilitation.The second study was a systematic review of the evidence for the effectiveness of nonpharmacological interventions, evaluating different existing rehabilitation programs for people with cardiac arrest.Based on the results of the two initial studies, the PhD student prepared a third study exploring the acceptability and feasibility of a rehabilitation intervention designed for survivors with cardiac arrest.Using an evidence-based research approach helped to lay the firm foundations for future randomized controlled trials.methods and design, thereby ensuring that the new study is valuable.
The justification process described previously is so closely related to the design process that in practice the two should not be performed separately.The justification process reaches a binary conclusion, namely the study is valuable, or it is not.If the new study does not add value, there is no reason to progress to the design process.If the new study is determined to be of value, the process leading to this conclusion is the same as for designing the new study.

Design informed by a systematic review
A systematic review will typically conclude with implications for practice and implications for research.In the newest version of the Cochrane Handbook (Chp 15.6, [46]), it is suggested to formulate the implications for research in relation to the grading of the evidence.A framework using a Population, Intervention, Comparison, Outcomes (PICO) approach to characterize important aspects of a possible evidence gap was published in 2011 [6].Table 2 gives an example of such evidence gaps identified through the synthesis of earlier studies in the Cochrane systematic review about aquatic exercise for knee and hip osteoarthritis from 2016 using Robinson's framework [6,66].

Reporting
An inevitable question is how the applied evidencebased research approach should be documented when the study is being written up for publication.CONSORT clearly recommends that the ''need for a new trial should be justified in the introduction.Ideally, it should include a reference to a systematic review of previous similar trials or a note of the absence of such trials'' [67].Even in the first version of CONSORT [68], this issue was emphasized butdas metaresearch is clearly indicatingdvery few authors are following this guidance [10,22,24,69e72] and very few journals are requesting, let alone enforcing it.Taking the example from Table 2, a new study aiming to meet the identified evidence gaps listed in Bartels' Cochrane review should refer to and cite this systematic review when providing a justification for the new study [66].
The introduction section could also be used to document the process for determining the relevant components of study design.In our example, authors might simply state with regard to selected primary clinical outcomes that ''there is moderate quality evidence that aquatic exercise may have small, short-term, and clinically relevant effects on patient-reported pain, disability, and QoL in people with knee and hip osteoarthritis'' [66].Or they might mention that the evidence has identified a number of gaps relating to disease severity when justifying the study's patient groups.Authors of a new study protocol should not only state the fact but also explain how the research question and the design of the new study are justified and informed by the perspectives of end users and the implications for research in a relevant systematic review.
In cases where a study protocol is published, this would be the optimal opportunity to report the evidence-based research approach to study design, documenting the reasoning around the study's PICO selection and the prioritization of the identified evidence gaps in relation to the clinical relevance for the patient and the practitioner in much greater detail than in the final article summarizing study results.One good example was a protocol for a systematic review on the impact of periodontal therapy on measures of disease activity and actual inflammatory burden in patients with rheumatoid arthritis.Its introduction section included an evidence-based research component providing a detailed description of the PICO question, the information sources used, the results of the search, and the implications of these results for the justification and design of the new systematic review about this topic.
If ethic committees and funding agencies demanded such an approach and level of information in the applications for ethical approval and financial support, this would create a strong incentive to systematically and transparently consider the relevance and necessity of any new study.Apart from a few exceptions [73], funding agencies have not yet implemented this requirement [74].

Discussion
Although by itself it does not address all issues required to demonstrate the value of new research, the evidencebased research approach is a crucial part of the process of justifying and designing a new study.
Proposing the evidence-based research approach to justify and design new studies, we recognize that there are a number of important challenges.First, as noted in the introduction to this series (REF to EBR article #1), many clinical researchersddespite being aware of systematic reviewsdlack the knowledge, skills, and time to conduct one.Strongly related to this first challenge is the current lack of incentive for researchers to prioritize the preparation of a systematic review over actually starting their new study.These challenges need to be tackled by the wider research ecosystem and not seen purely as the responsibility of the individual researcher.Research institutions and funding agencies, ethics committees, and publishers need to adapt and remove barriers to the implementation of an evidence-based research approach.The relevant training curricula and facilities need to be provided to equip researchers with the necessary skills to identify and use (and if need be update or conduct) systematic reviews, and so facilitate the use of systematic reviews when planning new studies.
Research waste due to the wrong questions being asked by scientists, poor study design, inaccessible research, and selective and biased reporting [75] is not only a matter of redundant studies with no apparent value for research, practice, or end users, but also perpetuates irresponsible use of funding resources and risks damaging the public's trust in research [76].
Therefore, how should these challenges of dissemination and anchoring of evidence-based research be met?We recommend structured and mandatory training in finding and critically appraising systematic reviews at undergraduate and postgraduate levels and the active empowerment of mentors and supervisors by offering relevant training to senior researchers during the transition period until the evidence-based research approach is fully established.Some might argue that our recommendations will place a considerable burden of effort and time on researchers, but as outlined in our series, the evidence-based research approach is an absolute necessity to ensure valuable research.The greatest waste and burden is to carry out a completely unnecessary research study!Table 2.The example of a Cochrane review on aquatic exercise for patients with knee and hip osteoarthritis [66] using Robinson's framework to identify evidence gaps [6]

Population (P) Intervention (I) Comparison (C) Outcome (O)
PICOPatients with knee and hip osteoarthritis: -Severity of disease (unclearly reported) -use international standard for defining a patient with knee and hip osteoarthritis Aquatic exercise: -clear type of aquatic exercise -design studies testi2ng different exercise doses relevant for clinical practice Control: -lack of studies comparing often used alternative treatments Pain, function, and quality of life were measured, but no studies have measured the effect of aquatic exercise on fatigue -measured immediately after the end of treatment -appropriate follow-up time

Fig. 1 .
Fig. 1.The evidence-based research approach highlighting the steps to be taken before a study is conducted.

Table 1 .
Key searchable resources for existing systematic reviews Joanna Briggs Institute EBP Database https://joannabriggs.org/ Health-related research questions relevant for allied health care professionals PROSPERO and Open Science Framework PROSPERO (https://www.crd.york.ac.uk/ prospero/) Open Science Framework (https://osf.io/)Registered systematic reviews currently in progress