Abstract
Objective
We established a large database of trials to serve as a resource for future methodological
and ethical analyses. Here, we use meta-data to describe the broad landscape of pragmatic
trials including research areas, identification as pragmatic, quality of trial registry
data and enrolment.
Study Design and Setting
Trials were identified by a validated search filter and included if a primary report
of a health-related randomized trial published January 2014-April 2019. Data were
collated from MEDLINE, Web of Science, ClinicalTrials.gov, and full text.
Results
4337 eligible trials were identified from 13,065 records, of which 1988 were registered
in ClinicalTrials.gov. Research areas were diverse, with the most common being general
and internal medicine; public, environmental and occupational health; and health care
sciences and services. The term “pragmatic” was seldom used in titles or abstracts.
Several domains in ClinicalTrials.gov had questionable data quality. We estimated
that one-fifth of trials under-accrued by at least 15%.
Conclusion
There is a need to improve reporting of pragmatic trials and quality of trial registry
data. Under accrual remains a challenge in pragmatic RCTs despite calls for more streamlined
recruitment approaches. The diversity of pragmatic trials should be reflected in future
ethical analyses.
Keywords
To read this article in full you will need to make a payment
Purchase one-time access:
Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online accessOne-time access price info
- For academic or personal research use, select 'Academic and Personal'
- For corporate R&D use, select 'Corporate R&D Professionals'
Subscribe:
Subscribe to Journal of Clinical EpidemiologyAlready a print subscriber? Claim online access
Already an online subscriber? Sign in
Register: Create an account
Institutional Access: Sign in to ScienceDirect
References
- Explanatory and pragmatic attitudes in therapeutical trials.J Chronic Dis. 1967; 20: 637-648https://doi.org/10.1016/j.jclinepi.2009.01.012
- A new mechanistic-practical framework for designing and interpreting randomized trials.J Clin Epidemiol. 2009; 62: 479-484https://doi.org/10.1016/j.jclinepi.2008.02.009
- Practical clinical trials: increasing the value of clinical research for decision making in clinical and health policy.JAMA. 2003; 290: 1624-1632
- Real-world evidence: How pragmatic are randomized controlled trials labeled as pragmatic?.BMC medicine. 2018; 16https://doi.org/10.1186/s12916-018-1038-2
- Making trials matter: pragmatic and explanatory trials and the problem of applicability.Trials. 2009; 10: 37https://doi.org/10.1186/1745-6215-10-37
- What kind of randomized trials do we need?.CMAJ. 2009; 180: 998-1000https://doi.org/10.1503/cmaj.082007
Patient-Centred Outcomes Research Institute (PCORI): PCORI Funding Announcement: Pragmatic Clinical Studies To Evaluate Patient-Centered Outcomes. Available at: https://www.pcori.org/sites/default/files/PCORI-PFA-2018-Cycle-1-Pragmatic-Studies.pdf Accessed 23 May 2018.
- A pragmatic view on pragmatic trials.Dialogues Clin Neurosci. 2011; 13: 217-224
- The PRECIS-2 tool: designing trials that are fit for purpose.BMJ. 2015; 350: h2147https://doi.org/10.1136/bmj.h2147
- Improving the reporting of pragmatic trials: an extension of the CONSORT statement.BMJ. 2008; 337: a2390https://doi.org/10.1136/bmj.a2390
- Ethics issues in pragmatic randomized controlled trials: a review of the recent literature identifies gaps in argumentation.BMC Med Ethics. 2018; : 14https://doi.org/10.1186/s12910-018-0253-x
- The ethical challenges raised in the design and conduct of pragmatic trials: an interview study with key stakeholders.Trials. 2019; 20: 765https://doi.org/10.1186/s13063-019-3899-x
- Series: Pragmatic trials and real world evidence: paper 1. introduction.J Clin Epidemiol. 2017; 88: 7-13https://doi.org/10.1016/j.jclinepi.2016.12.023
- Assessment of pragmatism in recently published randomized clinical trials.JAMA Intern Med. 2018; 178: 1278-1280https://doi.org/10.1001/jamainternmed.2018.3200
- Opportunities and barriers for pragmatic embedded trials: triumphs and tribulations.Learn Health Syst. 2017; https://doi.org/10.1002/lrh2.10044:e10044.e10044
- Patients and investigators prefer measures of absolute risk in subgroups for pragmatic randomized trials.J Clin Epidemiol. 2018; 103: 10-21https://doi.org/10.1016/j.jclinepi.2018.06.009
- Systematic review of pragmatic randomised control trials assessing the effectiveness of professional pharmacy services in community pharmacies.BMC Health Serv Res. 2021; 21: 156https://doi.org/10.1186/s12913-021-06150-8
- Developing a framework for the ethical design and conduct of pragmatic trials in healthcare: a mixed methods research protocol.Trials. 2018; 19: 525https://doi.org/10.1186/s13063-018-2895-x
- Cluster over individual randomization: are study design choices appropriately justified? Review of a random sample of trials.Clin Trials. 2020; 17: 253-263https://doi.org/10.1177/1740774519896799
- Inadequacy of ethical conduct and reporting of stepped wedge cluster randomized trials: Results from a systematic review.Clin Trials. 2017; 14: 333-341https://doi.org/10.1177/1740774517703057
- TwiC or treat? Are trials within cohorts ethically defensible?.Clinical Trials. 2018; 15: 21-24
- Ethics and practice of Trials within Cohorts: an emerging pragmatic trial design.Clin Trials. 2018; 15: 9-13https://doi.org/10.1177/1740774517746620
- A search filter to identify pragmatic trials in MEDLINE was highly specific but lacked sensitivity.J Clin Epidemiol. 2020; 124: 75-84
- A pragmatic resolution.J Clin Epidemiol. 2009; 62: 495-498https://doi.org/10.1016/j.jclinepi.2008.08.014
- The PRECIS-2 tool seems not to be useful to discriminate the degree of pragmatism of medicine masked trials from that of open-label trials.Eur J Clin Pharmacol. 2020; https://doi.org/10.1007/s00228-020-03030-8
- The design can limit PRECIS–2 retrospective assessment of the clinical trial explanatory/pragmatic features.J Clin Epidemiol. 2020; https://doi.org/10.1016/j.jclinepi.2020.03.027
- PRECIS-2 for retrospective assessment of RCTs in systematic reviews: some thoughts on intention, dichotomization and applicability of RCTs.J Clin Epidemiol. 2020; https://doi.org/10.1016/j.jclinepi.2020.06.023
- Study characteristics impacted the pragmatism of randomized controlled trial published in nursing: a meta-epidemiological study.J Clin Epidemiol. 2019; 116: 18-25https://doi.org/10.1016/j.jclinepi.2019.07.017
- The role of pragmatism in explaining heterogeneity in meta-analyses of randomised trials: a protocol for a cross-sectional methodological review.BMJ Open. 2017; 7e017887https://doi.org/10.1136/bmjopen-2017-017887
- Exploring the pragmatic and explanatory study design on outcomes of systematic reviews of public health interventions: a case study on obesity prevention trials.J Public Health (Oxf). 2014; 36: 170-176https://doi.org/10.1093/pubmed/fdu006
- Pragmatic vs. explanatory: an adaptation of the PRECIS tool helps to judge the applicability of systematic reviews for daily practice.J Clin Epidemiol. 2011; 64: 1095-1101https://doi.org/10.1016/j.jclinepi.2010.11.020
- The importance of decision intent within descriptions of pragmatic trials.J Clin Epidemiol. 2020; 125: 30-37
- "pragmatic trial": An essentially contested concept?.J Eval Clin Pract. 2019; https://doi.org/10.1111/jep.13216:1-12
- Covidence systematic review software.Melbourne, Australia2019
Clarivate Analytics: Research Areas (Categories /Classification) Available at: https://images.webofknowledge.com/images/help/WOS/hp_research_areas_easca.html. Accessed 24 March 2020.
- Adherence to the international committee of medical journal editors' (ICMJE) prospective registration policy and implications for outcome integrity: a cross-sectional analysis of trials published in high-impact specialty society journals.Trials. 2018; 19: 448https://doi.org/10.1186/s13063-018-2825-y
- Trends in global clinical trial registration: an analysis of numbers of registered clinical trials in different parts of the world from 2004 to 2013.BMJ Open. 2015; 5e008932https://doi.org/10.1136/bmjopen-2015-008932
- The landscape of clinical trials in nephrology: a systematic review of Clinicaltrials.gov.Am J Kidney Dis. 2014; 63: 771-780https://doi.org/10.1053/j.ajkd.2013.10.043
- Non-publication of large randomized clinical trials: cross sectional analysis.BMJ. 2013; 347: f6104https://doi.org/10.1136/bmj.f6104
- Characteristics of clinical trials registered in ClinicalTrials.gov, 2007-2010.JAMA. 2012; 307: 1838-1847
- Update on Trial Registration 11 Years after the ICMJE Policy Was Established.N Engl J Med. 2017; 376: 383-391https://doi.org/10.1056/NEJMsr1601330
Richardson L. beautifulsoup4 4.8.2. 4.8.2 edn; 2019.
Airtable. Available at: [https://airtable.com/product] Accessed 19 April 2021.
- Unsuccessful trial accrual and human subjects protections: an empirical analysis of recently closed trials.Clin Trials. 2015; 12: 77-83https://doi.org/10.1177/1740774514558307
- Differences between information in registries and articles did not influence publication acceptance.J Clin Epidemiol. 2015; 68: 1059-1067.https://doi.org/10.1016/j.jclinepi.2014.11.019
- Global mapping of randomised trials related articles published in high-impact-factor medical journals: a cross-sectional analysis.Trials. 2020; 21: 34https://doi.org/10.1186/s13063-019-3944-9
- Funding source, trial outcome and reporting quality: are they related? Results of a pilot study.BMC Health Serv Res. 2002; 2: 18
- Characteristics of funding of clinical trials: cross-sectional survey and proposed guidance.BMJ Open. 2017; 7e015997https://doi.org/10.1136/bmjopen-2017-015997
- The association of funding source on effect size in randomized controlled trials: 2013-2015 - a cross-sectional survey and meta-analysis.Trials. 2017; 18: 125https://doi.org/10.1186/s13063-017-1872-0
- Comparison of serious adverse events posted at ClinicalTrials.gov and published in corresponding journal articles.BMC Med. 2015; 13: 189https://doi.org/10.1186/s12916-015-0430-4
- The reporting of funding in health policy and systems research: a cross-sectional study.Health Res Policy Syst. 2018; 16: 83https://doi.org/10.1186/s12961-018-0356-3
- Assessing genitourinary cancer clinical trial accrual sufficiency using archived trial data.JCO Clinical Cancer Informatics. 2020; 4: 614-622
- Assessment of trends in the design, accrual, and completion of trials registered in clinicaltrials.gov by sponsor type, 2000-2019.JAMA Netw Open. 2020; 3e2014682https://doi.org/10.1001/jamanetworkopen.2020.14682
- Articles provided insufficient information to conduct an appropriate retrospective assessment of the pragmatic/explanatory features of medicine trials with the PRECIS-2 tool.Eur J Clin Pharmacol. 2020; https://doi.org/10.1007/s00228-020-02901-4
- The ClinicalTrials.gov results database — update and key issues.N Engl J Med. 2011; 364: 852-860
- Changes to registration elements and results in a cohort of Clinicaltrials.gov trials were not reflected in published articles.J Clin Epidemiol. 2016; 70: 26-37https://doi.org/10.1016/j.jclinepi.2015.07.007
- Comparison of registered and published outcomes in randomized controlled trials: a systematic review.BMC Medicine. 2016; https://doi.org/10.1186/s12916-015-0520-3
- Consistency of trial reporting between ClinicalTrials.gov and corresponding publications: one decade after FDAAA.Trials. 2020; 21https://doi.org/10.1186/s13063-020-04603-9
- Revisión crítica del ensayo clínico pragmático.Med Clin (Barc). 2003; 121: 384-388
- Ethical complexities in standard of care randomized trials: a case study of morning versus nighttime dosing of blood pressure drugs.Clin Trials. 2015; 12: 557-563https://doi.org/10.1177/1740774515607213
- Varieties of standard-of-care treatment randomized trials: ethical implications.JAMA. 2015; 313: 895-896https://doi.org/10.1001/jama.2014.18528
- Ethics of Informed Consent for Pragmatic Trials with New Interventions.Value Health. 2017; 20: 902-908https://doi.org/10.1016/j.jval.2017.04.005
- Ethical and regulatory issues of pragmatic cluster randomized trials in contemporary health systems.Clin Trials. 2015; 12: 276-286https://doi.org/10.1177/1740774515571140
- Exploring the ethical and regulatory issues in pragmatic clinical trials.Clin Trials. 2015; 12: 436-441https://doi.org/10.1177/1740774515598334
- Ethics and regulatory complexities for pragmatic clinical trials.JAMA. 2014; 311: 2381-2382https://doi.org/10.1001/jama.2014.4164
- Health policy and systems research: towards a better understanding and review of ethical issues.BMJ Global Health. 2017; 2e000314https://doi.org/10.1136/bmjgh-2017-000314
- Low risk pragmatic trials do not always require participants' informed consent.BMJ. 2019; 364: l1092https://doi.org/10.1136/bmj.l1092
- Informed Consent for Pragmatic Trials — The Integrated Consent Model.N Engl J Med. 2014; 370: 769-772
- Participant eligibility, recruitment, and retention in pragmatic trials.J Clin Epidemiol. 2017; 89: 173-180https://doi.org/10.1016/j.jclinepi.2016.12.021
Article info
Publication history
Published online: March 28, 2021
Accepted:
March 18,
2021
Footnotes
Conflicts of interest: CW receives consulting income from Cardialen, Eli Lilly & Company, and Research Triangle Institute (RTI) International.
Identification
Copyright
© 2021 Elsevier Inc. All rights reserved.
