Advertisement

A meta-research study revealed several challenges in obtaining placebos for investigator-initiated drug trials

  • Benjamin Speich
    Correspondence
    Corresponding author. Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Windmill Road, OX3 7LD, Oxford, United Kingdom.
    Affiliations
    Nuffield Department of Orthopaedics, Oxford Clinical Trials Research Unit and Centre for Statistics in Medicine, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford, United Kingdom

    Department of Clinical Research, Basel Institute for Clinical Epidemiology and Biostatistics, University Hospital Basel, University of Basel, Basel, Switzerland
    Search for articles by this author
  • Patricia Logullo
    Affiliations
    Nuffield Department of Orthopaedics, Oxford Clinical Trials Research Unit and Centre for Statistics in Medicine, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford, United Kingdom

    The EQUATOR Network, Oxford, United Kingdom
    Search for articles by this author
  • Stefanie Deuster
    Affiliations
    Hospital Pharmacy, University Hospital Basel, Basel, Switzerland
    Search for articles by this author
  • Ioana R. Marian
    Affiliations
    Nuffield Department of Orthopaedics, Oxford Clinical Trials Research Unit and Centre for Statistics in Medicine, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford, United Kingdom
    Search for articles by this author
  • Joanna Moschandreas
    Affiliations
    Nuffield Department of Clinical Neurosciences, Centre for the Prevention of Stroke and Dementia, University of Oxford, Oxford, United Kingdom
    Search for articles by this author
  • Ala Taji Heravi
    Affiliations
    Department of Clinical Research, Basel Institute for Clinical Epidemiology and Biostatistics, University Hospital Basel, University of Basel, Basel, Switzerland

    Swiss Tropical and Public Health Institute, Basel, Switzerland
    Search for articles by this author
  • Viktoria Gloy
    Affiliations
    Department of Clinical Research, Basel Institute for Clinical Epidemiology and Biostatistics, University Hospital Basel, University of Basel, Basel, Switzerland
    Search for articles by this author
  • Matthias Briel
    Affiliations
    Department of Clinical Research, Basel Institute for Clinical Epidemiology and Biostatistics, University Hospital Basel, University of Basel, Basel, Switzerland

    Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, Canada
    Search for articles by this author
  • Sally Hopewell
    Affiliations
    Nuffield Department of Orthopaedics, Oxford Clinical Trials Research Unit and Centre for Statistics in Medicine, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford, United Kingdom
    Search for articles by this author
  • for theMAking Randomized Trials Affordable (MARTA) Group
Open AccessPublished:November 23, 2020DOI:https://doi.org/10.1016/j.jclinepi.2020.11.007

      Abstract

      Objectives

      To systematically assess the kind of placebos used in investigator-initiated randomized controlled trials (RCTs), from where they are obtained, and the hurdles that exist in obtaining them.

      Study Design and Setting

      PubMed was searched for recently published noncommercial, placebo-controlled randomized drug trials. Corresponding authors were invited to participate in an online survey.

      Results

      From 423 eligible articles, 109 (26%) corresponding authors (partially) participated. Twenty-one of 102 (21%) authors reported that the placebos used were not matching (correctly labeled in only one publication). The main sources in obtaining placebos were hospital pharmacies (32 of 107; 30%) and the manufacturer of the study drug (28 of 107; 26%). RCTs with a hypothesis in the interest of the manufacturer of the study drug were more likely to have obtained placebos from the drug manufacturer (18 of 49; 37% vs. 5 of 29; 17%). Median costs for placebos and packaging were US$ 58,286 (IQR US$ 2,428– US$ 160,770; n = 24), accounting for a median of 10.3% of the overall trial budget.

      Conclusion

      Although using matching placebos is widely accepted as a basic practice in RCTs, there seems to be no standard source to acquire them. Obtaining placebos requires substantial resources, and using nonmatching placebos is common.

      Graphical abstract

      Keywords

      What is new?

        Key findings

      • Of the used placebos, 21% (21 of 102) did not match the intervention drug being tested that was almost never reported in the corresponding publication.
      • The original manufacturer seemed to be more likely to support investigator-initiated studies (with drugs and placebos) when the study hypothesis was in their interest.
      • Of the contacted investigators, 6% (6 of 101) already canceled a trial due to lack of placebos, and 27% (27 of 101) had to adapt the study design.
      • The median costs to obtain placebos was US$ 58,286, accounting for a median of 10.3% of the total trial budget.

        What this adds to what is known

      • Obtaining matching placebos for investigator-initiated drug trials can be burdensome, which hampers the conduct of nonindustry initiated randomized controlled trials.

        What is the implication, and what should change now?

      • Clear official sources to obtain matching placebos should be available.
      • Open reporting of the use of nonmatching placebos in research articles should be encouraged.

      1. Introduction

      Methodologically sound and well-conducted randomized controlled trials (RCTs) provide the most trustworthy evidence and are, therefore, the gold standard when evaluating the efficacy and harms of a new medical intervention [
      • Duley L.
      • Antman K.
      • Arena J.
      • Avezum A.
      • Blumenthal M.
      • Bosch J.
      • et al.
      Specific barriers to the conduct of randomized trials.
      ,
      • Collins R.
      • MacMahon S.
      Reliable assessment of the effects of treatment on mortality and major morbidity, I: clinical trials.
      ]. RCTs are typically conducted by one of two parties [
      • Suvarna V.
      Investigator initiated trials (IITs).
      ,
      • Konwar M.
      • Bose D.
      • Gogtay N.J.
      • Thatte U.M.
      Investigator-initiated studies: challenges and solutions.
      ]. First are industry-initiated trials conducted to receive approval of a new medical product (e.g., drugs, medical devices). The second is group trials initiated by academic researchers that are not sponsored by industry; so-called investigator-initiated trials. These investigator-initiated trials are crucial to answer clinical questions, which do not have a direct prospect for financial profit [
      • Konwar M.
      • Bose D.
      • Gogtay N.J.
      • Thatte U.M.
      Investigator-initiated studies: challenges and solutions.
      ]. Examples of research questions without any direct prospect for financial profit include: (i) whether a shorter duration of antimicrobial treatment is noninferior to a longer treatment [
      • Speich B.
      • Bausch K.
      • Roth J.A.
      • Hemkens L.G.
      • Ewald H.
      • Vogt D.R.
      • et al.
      Single-dose versus 3-day cotrimoxazole prophylaxis in transurethral resection or greenlight laser vaporisation of the prostate: study protocol for a multicentre randomised placebo controlled non-inferiority trial (CITrUS trial).
      ], or (ii) comparing drugs for a disease which are marketed by different pharmaceutical companies [
      • Han Y.
      • Guo J.
      • Zheng Y.
      • Zang H.
      • Su X.
      • Wang Y.
      • et al.
      Bivalirudin vs heparin with or without tirofiban during primary percutaneous coronary intervention in acute myocardial infarction: the BRIGHT randomized clinical trial.
      ].
      Blinding can control a variety of potential biases when conducting RCTs. In the case of drug trials, blinding is generally achieved by using placebos that are identical to the drug under investigation in aspects such as color, shape, weight, smell, and taste but do not contain the active compound [
      • Wan M.
      • Orlu-Gul M.
      • Legay H.
      • Tuleu C.
      Blinding in pharmacological trials: the devil is in the details.
      ]. The practical hurdles of obtaining access to placebos for the conduct of investigator-initiated drug trials are often underestimated [
      • Wan M.
      • Orlu-Gul M.
      • Legay H.
      • Tuleu C.
      Blinding in pharmacological trials: the devil is in the details.
      ,
      • Whitham D.
      • Silcocks P.
      • Whitehouse W.
      • Hodgson S.
      • Sammons H.
      An example of problems that arise from clinical trials and how to avoid them.
      ]. Probably the easiest and ideal solution would be if matching placebos were provided by the pharmaceutical company producing the drug that is assessed in the planned investigator-initiated RCT. However, pharmaceutical companies often have limited interest in providing placebos, and some instances are documented where they only agreed to provide the placebo if the trial protocol was changed following their review [
      • Wan M.
      • Orlu-Gul M.
      • Legay H.
      • Tuleu C.
      Blinding in pharmacological trials: the devil is in the details.
      ,
      • Curfman G.D.
      • Morrissey S.
      • Drazen J.M.
      Products at risk.
      ,
      • Christensen M.
      • Knop F.K.
      The unobtainable placebo: control of independent clinical research by industry?.
      ].
      The alternative is usually to obtain placebos from other sources. This solution can introduce new difficulties. For example, the placebo provided by a different manufacturer may not match the original drug in all aspects (e.g., engravings, color, form, taste). Another solution that has been proposed for orally administered tablets is overencapsulating the placebo, as well as the drug that is tested in the RCT. Some argue that this is not ideal, as it increases the size of the tablet (might be difficult to swallow for the patient), and in the end, it creates a different formulation that may have different effects [
      • Martin B.K.
      • Meinert C.L.
      • Breitner J.C.
      • Group A.R.
      Double placebo design in a prevention trial for Alzheimer's disease.
      ].
      In general, when placebos are acquired from a different source to the original drug manufacturer, repackaging of the original drug to ensure matching and blinding becomes inevitable. This again is problematic as the guaranteed drug expiry date provided by the manufacturer is only valid in the original package [
      • Wan M.
      • Orlu-Gul M.
      • Legay H.
      • Tuleu C.
      Blinding in pharmacological trials: the devil is in the details.
      ].
      Overall, although matching placebos are crucial for RCTs on pharmacological treatments, how to obtain them is often not straightforward, and it can remain unclear in the publication of RCT results if the placebos were really matching [
      • Wan M.
      • Orlu-Gul M.
      • Legay H.
      • Tuleu C.
      Blinding in pharmacological trials: the devil is in the details.
      ,
      • Curfman G.D.
      • Morrissey S.
      • Drazen J.M.
      Products at risk.
      ,
      • Christensen M.
      • Knop F.K.
      The unobtainable placebo: control of independent clinical research by industry?.
      ]. According to our knowledge, no systematic assessment has been done to find out the kind of placebos that are used and the hurdles that researchers face when using placebos in investigator-initiated drug trials.
      Therefore, we identified recently conducted investigator-initiated drug trials to assess the kind of placebos that are used. Furthermore, we contacted the corresponding author of each trial to find out more about the specific characteristics of the placebo used and about any hurdles encountered when obtaining the placebo.

      2. Methods

      2.1 Sample selection and eligibility criteria

      We systematically searched PubMed for placebo-controlled RCTs published between March 1, 2018, and February 28, 2019, with the search conducted on March 6, 2019. The search was repeated to increase the sample of survey responses on September 12, 2019, searching for placebo-controlled RCTs published between March 1, 2018, and August 31, 2019 (see detailed search strategy in Appendix). We did not search for older publications as we intended to assess the current situation, and we assumed that corresponding authors of older RCTs are less likely to respond.
      Eligible RCTs for this meta-research study needed to fulfill the following inclusion criteria: (i) the intervention under investigation needed to be a medicinal product (i.e., a drug); (ii) the identified RCT must have randomized some participants to receive a placebo; (iii) the publication needed to present the primary results (i.e., no indication that the publication is not the main-publication” of the RCT); and (iv) the RCT must have been investigator-initiated (i.e., nonindustry sponsored, which was simplified for practical reasons to RCTs with nonindustry funding and nonindustry involvement). RCTs in which a pharmaceutical company provided the drugs only but were funded by academia, institutional, or governmental funding agencies were still eligible. We included RCTs published in English, German, French, Portuguese, Spanish, and Italian.
      We excluded RCTs that were clearly labeled as pilot or feasibility studies, pharmacokinetic studies, studies with healthy volunteers, and RCTs not including humans. Furthermore, we excluded RCTs testing the effect of dietary supplements (as defined by the Food and Drug Administration), traditional herbal medicine (or herbalism), phytotherapy or any plant extracts, traditional Chinese/Asian medicine, homeopathy, or biological products (e.g., vaccines, immunotherapy) or biologically similar synthetic products. We also excluded RCTs without a full-text publication and articles presenting results of more than one RCT.

      2.2 Screening of potentially relevant articles

      One reviewer (BS) excluded all obviously ineligible studies based on the title and abstract. For the potentially eligible publications, we obtained the corresponding full texts, which were screened by two reviewers (BS, PL, IRM, JM, SD, ATH, VG). The funding statement and the declaration of interests section were assessed to identify if a trial was industry-initiated or investigator-initiated. Based on the full text, the final decision was made on whether an article was otherwise eligible.

      2.3 Ethics, data extraction, and questionnaire

      This study was approved by the Medical Sciences Interdivisional Research Ethics Committee of the University of Oxford (R64131/RE001). From all eligible RCTs, we extracted the following information in duplicate (conducted by; BS, PL, IRM, JM, SD, ATH, VG): number of participants randomized, age group of the patient population (i.e., children, adults, elderly), number of study centers, number of treatment arms, study design (e.g., parallel-arm, cross-over, split body), dosage form (i.e., tablet, capsule, liquid [oral], liquid [injection], cream, other). We also looked at any evidence that the manufacturer of the drug was partially involved in the study (i.e., providing drugs or placebo), the clinical trial phase (e.g., phase I, II, III, or IV), name and email address of the corresponding author. Conflicts in the data extraction were resolved by discussion.
      An online questionnaire was created using a tool developed by SoSciSurvey (https://www.soscisurvey.de/). We invited all corresponding authors of RCTs by email to participate in the online survey (see invitation email in the Appendix). In the invitation, we inserted the title and the link to the identified full-text publication to make sure that the responses could be linked to those articles. Corresponding authors were informed that participation would take about 5–10 minutes and that only aggregate data would be published (i.e., no individual responses).
      The questionnaire (see Appendix for full questionnaire) contained specific questions covering the following topics: who provided the placebo; how matching was accomplished; were placebos really matching; if and how the drugs were repacked; how difficult it was to obtain the placebo (on a scale from 0 to 10); how satisfied they were with the placebos used (on a scale from 0 to 10); the costs of the placebo; whether they thought that their study hypothesis was in the interest of the drug manufacturer; whether they had to modify or cancel a previous trial because they could not obtain a matching placebo. Additionally, they could make any comments at the end of the questionnaire.
      The questionnaire was pre-tested by study team members (BS, SD, JM, MB, and SH) and refined accordingly. Questionnaires were sent out in July 2019 for eligible RCTs published between 1 March 2018 and 28 February 2019 and in November 2019 for eligible published RCTs identified in the updated search. Up to four reminders were sent every 2–3 weeks after the original invitation.

      2.4 Analysis

      Trial characteristics and questionnaire responses were summarized descriptively. Due to the exploratory nature of this study, no formal statistical hypothesis testing was used. Continuous data were presented as median and interquartile ranges (IQR; 25th and 75th percentiles). Since not all questions are answered by the same number of corresponding authors, both the numerator and denominator (and corresponding percentage) for each survey question were reported. Subgroups were created to separately analyze questionnaire responses for placebos of (a) tablets or capsules and (b) oral or injected liquids. Studies that obtained placebos from the drug manufacturer were also analyzed separately to those that obtained placebos from other sources. Odds ratios and 95% confidence intervals (CI) were calculated to assess if studies with a hypothesis in the interests of the drug manufacturers were more likely to receive support from the manufacturer. We excluded reported overall trial costs that we judged as clearly unrealistic (i.e., ≤ US$ 2,500, an amount that (i) barely covers ethical approval charges in some countries, (ii) nor the preparation and planning costs of an RCT [
      • Speich B.
      • Schur N.
      • Gryaznov D.
      • von Niederhausern B.
      • Hemkens L.G.
      • Schandelmaier S.
      • et al.
      Resource use, costs, and approval times for planning and preparing a randomized clinical trial before and after the implementation of the new Swiss human research legislation.
      ], and (iii) which also seems unrealistically low to cover the costs for RCTs conducted in low-income countries [
      • Speich B.
      • von Niederhausern B.
      • Blum C.A.
      • Keiser J.
      • Schur N.
      • Furst T.
      • et al.
      Retrospective assessment of resource use and costs in two investigator-initiated randomized trials exemplified a comprehensive cost item list.
      ]).

      2.5 Reporting and protocol

      This study was written in adherence to the Guidelines for reporting meta-epidemiological methodology research” [
      • Murad M.H.
      • Wang Z.
      Guidelines for reporting meta-epidemiological methodology research.
      ] and Good practice in the conduct and reporting of survey research” [
      • Kelley K.
      • Clark B.
      • Brown V.
      • Sitzia J.
      Good practice in the conduct and reporting of survey research.
      ] (see supplementary file). The study protocol is available as a supplementary file.

      3. Results

      3.1 Study characteristics

      The electronic search yielded 4,974 publications, of which 423 met the eligibility criteria (Figure 1); 420 studies were reported in English and one in each of French, Spanish, and Portuguese. A large proportion of studies used a tablet or capsule (n = 164; 38.8%) or a liquid injection as a medical compound (n = 131; 31.0%; Table 1). The median sample size was 87 patients (IQR, 52.5–154), and most RCTs were single-center studies (n = 250; 59.1%). Most used a parallel-group design (n = 380; 89.8%) and had two treatment arms (n = 336; 79.4%). Of the 423 studies, only 47 (11.1%) defined a trial phase in the publication, and for 67 studies (15.8%), it was reported that the manufacturer provided the drug or placebo. From the 423 publications, 207 (48.9%) reported whether the placebo was matching, 199 (47.0%) described what the placebo consisted of (e.g., saline, starch), and only 65 (15.4%) reported both. For tablets and capsules, it was usually described if they were matching (124 of 164; 76%) but not what they consisted of (27 of 164; 16%; Table S1, appendix). The opposite was true for liquid-based placebos (injection and oral). For liquids, it was rarely reported if they were matching (38 of 147; 26%), but relatively often what they consisted of (e.g., saline; 116 of 147; 79%).
      Figure thumbnail gr1
      Fig. 1Flow chart of included articles and participation in the survey. Abbreviations: RCT: randomized controlled trial.
      Table 1Characteristics of included randomized controlled trials extracted from published articles
      CharacteristicsEligible RCTs (n = 423)Participated in survey (n = 109)
      Form of the investigated drug
       Tablet, capsule164 (38.8%)54 (49.5%)
       Liquid injection131 (31.0%)23 (21.1%)
       Inhaler or nasal application25 (5.9%)5 (4.6%)
       Liquid oral16 (3.8%)7 (6.4%)
       Cream13 (3.1%)2 (1.8%)
       Other37
      Multiple forms of medicinal products (n = 6), vaginal application (n = 3), mouthwash (n = 2), and administration directly in the intra-abdominal space or open tissue during surgery (n = 1).
      (8.7%)
      12
      Multiple forms of medicinal products (n = 16), vaginal application (n = 7), eye drops (n = 5), administration directly in the intra-abdominal space or open tissue during surgery (n = 4), mouthwash (n = 2), patch on skin (n = 1), and spray on skin (n = 1).
      (11.0%)
       Unclear37
      14 stated that drug was given orally.
      (8.7%)
      6
      2 stated that drug was given orally.
      (5.5%)
      Sample size
       Median (IQR)87 (52.5–154)90 (54–237)
       Mean (SD)407 (1,686)705 (2515)
       <100238 (56.3%)60 (55.0%)
       100–200104 (24.6%)21 (19.3%)
       >20081 (19.1%)28 (25.7%)
      Population
       Adults365 (86.3%)91 (82.6%)
       Pediatric (<18 years)41 (9.7%)12 (11.0%)
       Only elderly (≥60 years)12 (2.8%)4 (3.7%)
       Children and adults5 (1.2%)3 (2.8%)
      Study centers
       Single center250 (59.1%)59 (54.1%)
       Multiple centers129 (30.5%)41 (37.5%)
       Unclear44 (10.4%)9 (8.3%)
      Trial design
       Parallel380 (89.8%)98 (89.9%)
       Crossover31 (7.3%)7 (6.4%)
       Factorial8 (1.9%)3 (2.8%)
       Split-body3 (0.7%)0 (0%)
       Cluster1 (0.2%)1 (0.9%)
      Number of treatment arms
       2 arms336 (79.4%)89 (81.7%)
       3 arms59 (13.9%)10 (9.2%)
       4 arms21 (5.0%)7 (6.4%)
       >4 arms7 (1.7%)3 (2.8%)
      Phase of trial (as reported in publication)
       Phase 221 (5.0%)7 (6.4%)
       Phase 325 (5.9%)11 (10.0%)
       Phase 41 (0.2%)0 (0%)
       Not reported376 (88.9%)91 (83.5%)
      Any evidence that manufacturer provided drug and placebo
       Yes67 (15.8%)29 (26.6%)
       No356 (84.2%)80 (73.4%)
      Abbreviations: RCT: randomized controlled trial; IQR: interquartile range, SD: Standard deviation.
      a Multiple forms of medicinal products (n = 6), vaginal application (n = 3), mouthwash (n = 2), and administration directly in the intra-abdominal space or open tissue during surgery (n = 1).
      b Multiple forms of medicinal products (n = 16), vaginal application (n = 7), eye drops (n = 5), administration directly in the intra-abdominal space or open tissue during surgery (n = 4), mouthwash (n = 2), patch on skin (n = 1), and spray on skin (n = 1).
      c 14 stated that drug was given orally.
      d 2 stated that drug was given orally.
      A total of 109 (25.8%) corresponding authors started filling out the survey, and 101 (23.9%) reached the last question (Figure 1). Overall, considering the assessed trial characteristics from Table 1, the RCTs of survey participants seem to be a representative sample of the 423 eligible RCTs (Table 1).

      3.2 Placebos used and repackaging

      The form of the compounds used, as reported by the investigators in the survey, is listed in Table 2. A median of 1,240 (IQR 81–9,200) placebos per study was used (Table 2). Seventy-nine of 107 corresponding authors answered that they did not obtain the placebo from the manufacturer of the investigated drug. When asked what kind of placebos were obtained, they reported the following: 25 used a capsule (16 stated clearly that the drug and placebo were over encapsulated; 1 reported that capsules were emptied and refilled with placebo); 22 used saline; 17 reported to use similar or matching tablets. Table S2 in the appendix lists other methods of achieving matching placebos.
      Table 2Authors’ responses to questions from online survey
      In which form was the placebo-matched drug given?N = 109
       Tablet36 (33%)
       Capsule26 (24%)
       Liquid38 (35%)
       Cream/gel2 (2%)
       Other7
      Multiple forms of medicinal product (n = 3); powder to dissolve (n = 2); skin patch (n = 1); pessary (n = 1).
      (6%)
      Which was the route of administration of the investigated drug?N = 108
       Oral66 (61%)
       Parenteral (including intravenous, intramuscular, subcutaneous, intradermal)24 (22%)
       Cutaneous application3 (3%)
       Inhalation5 (5%)
       Other10
      Vaginal (n = 3); multiple routes of administration (n = 2), topical oral (n = 1), buccally absorbed (n = 1), sublingual (n = 1), mouthwash (n = 1), fascial spaces (n = 1).
      (9%)
      Was the drug under investigation provided free of charge?N = 108
       No74 (69%)
       Yes34 (31%)
      How many placebo dosages were needed for the trial?N = 94
       Median (IQR)1,240 (81–9,200)
       (Minimum, maximum)(15–46,000,000
      Were drugs and placebo repacked to ensure blinding?N = 78
       Yes65 (83%)
       No13 (17%)
      Due to the repacking: was a reduction of the medicine expiry date taken into consideration?N = 63
       Yes34 (54%)
       No13 (21%)
       I don’t know11 (17%)
       Other5 (8%)
       Drugs were consumed very timely,” meaning long before expiration date3
      Expiry date was considered”1
       Drug stability was tested in new packaging1
      On a scale from 0–10, how complicated was it to obtain (matching placebos (0 = not complicated at all; 10 = could not have been more complicated)?N = 101
       Median (IQR); Mean (SD)3 (1–6); 3.6 (3.3)
      On a scale from 0–10, how content were you with the finally used placebos (0 = very disappointed; 10 = very satisfied with placebos)?N = 102
       Median (IQR); Mean (SD)10 (8–10); 9.8 (1.7)
      Considering also previous projects, did you ever have to cancel a study because you could not obtain matching placebos?N = 101
       Yes6 (6%)
       No95 (94%)
      Considering also previous projects, did you ever have to modify the design of a study because you could not obtain matching placebos?N = 101
       Yes27 (27%)
       No74 (73%)
      Abbreviations: IQR: interquartile range.
      a Multiple forms of medicinal product (n = 3); powder to dissolve (n = 2); skin patch (n = 1); pessary (n = 1).
      b Vaginal (n = 3); multiple routes of administration (n = 2), topical oral (n = 1), buccally absorbed (n = 1), sublingual (n = 1), mouthwash (n = 1), fascial spaces (n = 1).
      Of the 78 investigators who responded to the question on repackaging, 65 stated that the placebos and drugs were repacked to ensure blinding (Table 2). During this process, 34 of 63 investigators (54%) stated that a reduction in the expiry date was taken into consideration. The others declared that they did not do so (n = 13; 21%), did not know about it (n = 11; 17%); or gave other statements (e.g., that drugs were consumed very timely after repackaging; Table 2).

      3.3 Were placebos matching, and what was reported in the publication?

      The question if placebos were matching was answered by 102 corresponding authors (Table 2); 21 (21%) answered that the used placebos were not matching, and 8 (8%) were unsure if placebos were matching. None of the 21 investigators reported that the study design had to be changed (e.g., open label instead of blinding) due to nonmatching placebos. From the 21 RCTs in which the use of nonmatching placebos was reported by the investigators, only one corresponding publication (5%) declared this correctly. In 13 publications (62%), it was not reported if placebos were matching, while 7 (33%) even misleadingly labeled them as matching (Table 3). In the eight publications for which the corresponding authors answered that they were unsure if the placebos were matching, four publications declared them as matching, while four did not mention anything in this regard.
      Table 3Matching of placebos and study drugs: results of the survey with corresponding authors and assessment of adequate reporting in the publications
      Matching of placebosAll compoundsOnly tablets, pills and capsules
      Placebo matching?(n = 102)(n = 58)
       No21 (21%)9 (16%)
       Yes73 (72%)45 (78%)
       Unsure8 (8%)4 (7%)
      Reporting in publication of not matching placebos(n = 21)(n = 9)
       Correctly labeled1 (5%)0 (0%)
       Not labeled as nonmatching13 (62%)5 (56%)
       Misleadingly labeled (e.g., as “matching” or “ identical”)7 (33%)4 (44%)

      3.4 Source of placebo

      Thirty-two of 107 corresponding authors (30%) reported obtaining their placebos from a hospital pharmacy and 28 (26%) accessed their placebo through the manufacturer of the original drug. Additional sources were other pharmaceutical manufacturers (n = 19, 18%), pharmacies (n = 15; 14%), using simple available saline (n = 3; 3%), pharmaceutical department (n = 3; 3%), were self-made” (n = 2; 2%), or used other sources (see Table S3, appendix). Of the 79 investigators who did not obtain the placebo from the drug manufacturer, 22 answered, that they approached the manufacturer but could not obtain a matching placebo. The most common reasons were (multiple reasons could be listed): the manufacturer could not provide a placebo (n = 7; 32%) (e.g., limited resources, experience); the manufacturers were not interested (n = 6; 27%); costs were too high (n = 4; 18%); patent expired so the manufacturer was not interested (n = 3; 14%; see Appendix Table S4). The most common reasons why the other 57 investigators did not contact the original manufacturer were the following (multiple reasons could be listed): not needed (n = 15; 26%); investigators wanted to remain independent (n = 6; 11%); investigator was aware that manufacturer is not interested (n = 5; 9%); generic manufacturer existed (n = 5; 9%); placebo was simple (e.g., sterile saline; n = 4; 7%); costs (n = 3; 5%; see Appendix Table S4).

      3.5 Industry involvement

      Of the 28 investigators who reported that the manufacturer provided the placebo, most manufacturers requested to see the study protocol (19; 68%) and statistical analysis plan (18 of 28; 64%). However, very few requested changes (1 of 19 in the protocol; 2 of 18 in the analysis plan or subsequent report and publication); and of the requested changes, two investigators stated they declined any changes, and one stated that changes were implemented (Appendix Table S5).
      Forty-nine of the 101 corresponding authors stated that their original study hypothesis was in the interest of the drug manufacturer (e.g., evaluating if the drug is also effective in another indication). Twenty-nine declared that the study hypothesis was not in the manufacturer’s interest (e.g., evaluating if a cheaper alternative is as effective), 21 were unsure, and two marked that they did not want to answer this question. Studies with a hypothesis in the interest of the original manufacturer appeared more likely to obtain placebos from the drug manufacturer (18 of 49; 37%) compared to RCTs for which the hypothesis was not in the manufacturer’s interest (5 of 29; 17%; odds ratio: 2.79; 95% CI 0.90–8.59). Also, drugs were provided more often free of charge for studies, which had a hypothesis in the interests of the manufacturer (23 of 49; 47% vs. 5 of 29; 17%; odds ratio: 4.25; 95% CI 1.39–12.95). In total, 34 investigators answered that they received the drug under investigation for free from a pharmaceutical company. In 22 of these 34 (65%) publications, we found reported evidence that drugs were provided, while in 12 (35%) publications, this was not reported.

      3.6 Satisfaction with placebos

      On a scale of 0 to 10, most investigators responded that they were very satisfied (median 10; IQR 8–10; Table 2) with the placebo used and that the process to obtain placebos was considered relatively uncomplicated (median 3; IQR 1–6; Table 2).

      3.7 Costs

      Twenty-four investigators reported the costs of placebos (including packaging). Median costs of placebo were US$ 58,286 (IQR US$ 2,428–US$ 160,770) per RCT, which took up a relatively large amount of the overall budget of those trials (median 10.3%; IQR 2.8–22.0%). Median costs of placebos per included patient were US$ 142 (IQR US$ 25–US$ 390). Trial costs for an entire RCT were described by 77 investigators; of those, 18 reported highly unrealistic numbers. Hence, only 59 of the 77 answers were considered in our analysis. The median cost for an RCT was US$ 665,000 (IQR US$ 100,000–US$ 2,359,000). Median costs per patient were US$ 2,750 (IQR US$ 488–US$ 8918).

      3.8 Stopping or modifying a planned trial

      Investigators were also asked if they had to cancel or modify this trial or any previous ones because they could not obtain a matching placebo. Six reported that they had canceled a trial, and 27 reported that they had to modify the study design (Table 2).

      3.9 Additional comments from corresponding authors

      At the end of the survey, investigators were given the option to add general comments. Thirteen used this possibility to report difficulties in obtaining placebos. Challenges included: the cost of placebos or packaging (i.e., study could not be funded due to high placebo costs), issues in interacting with drug companies, and problems with ingestion of capsules because large capsules had to be used for overencapsulation. It was also suggested that a portal or a website containing different possible placebo manufacturers would be helpful (Appendix Table S6).

      4. Discussion

      In a publication from 2013, Wan and colleagues stated that the practical aspects of establishing blinding in investigator-initiated pharmacological trials are often grossly underestimated” [
      • Wan M.
      • Orlu-Gul M.
      • Legay H.
      • Tuleu C.
      Blinding in pharmacological trials: the devil is in the details.
      ]. In response to that, and because case reports describe that acquiring placebos is not always possible and can be very costly [
      • Curfman G.D.
      • Morrissey S.
      • Drazen J.M.
      Products at risk.
      ,
      • Christensen M.
      • Knop F.K.
      The unobtainable placebo: control of independent clinical research by industry?.
      ], we conducted this meta-research study. According to our knowledge, the present study is the first to systematically assess in the frame of a survey the kind of placebos used, where they are obtained, and the hurdles faced in obtaining them in investigator-initiated drug trials.
      Our study shows that approximately 20% of placebos used in investigator-initiated drug trials do not match the corresponding drugs (Graphical abstract Summary of key results”). This was usually not reported in the published article, or even worse, were misclassified as matching. Inadequate reporting of placebo controls was also found by Webster and colleagues, although not focusing on the reporting of matching [
      • Webster R.K.
      • Howick J.
      • Hoffmann T.
      • Macdonald H.
      • Collins G.S.
      • Rees J.L.
      • et al.
      Inadequate description of placebo and sham controls in a systematic review of recent trials.
      ].
      Our data imply that studies with a hypothesis in the interests of the drug manufacturer were more than two times as often supported with free drugs or provided with matching placebos. These results may indicate that providing placebos and the drug under investigation can be a powerful tool for manufacturers to support the conduct of trials which are in their interest. This is also supported by the fact that investigators who obtained placebos from a manufacturer declared that the manufacturer usually wanted to see the study protocol. Although changes were rarely requested, this shows that manufacturers are usually well aware of the study designs when they consider supporting investigator-initiated trials. It has to be noted that the present analysis was limited in size and that the 95% CI of the odds ratios were wide.
      When placebos were not provided by the manufacturer, they were usually acquired through hospital pharmacies or other manufacturers or pharmacies. Encapsulation, using saline or similar/matching tablets were the most popular solutions to match the drug under investigation. However, these rather simple solutions are not possible for all drugs. Our study showed that not being able to obtain a placebo had caused some investigators to terminate their planned trial (6 of 101) or to modify the trial design (27 of 101), meaning that the RCT was probably not conducted as originally planned. We are not aware of any similar studies to which we could compare these data.
      Acquiring placebos, including repacking, requires a substantial part of the overall trial budget; data from our survey suggests that this could be as much as 10% of the overall trial costs. This is in line with the case report from Curfman and colleagues, stating that the costs for placebos were US$ 900,000 [
      • Curfman G.D.
      • Morrissey S.
      • Drazen J.M.
      Products at risk.
      ]. Christensen and Knop also reported that a drug company charged an extraordinary amount of money for providing a simple placebo tablet, effectively preventing the planned clinical trial from going ahead” and that in another case, pharmaceutical companies refused to deliver placebos without stating any reasons [
      • Christensen M.
      • Knop F.K.
      The unobtainable placebo: control of independent clinical research by industry?.
      ]. The fact that it can be burdensome to obtain matching placebos was highlighted by several corresponding authors who specifically reported problems in obtaining placebos in the open comment field. In contrast to that, investigators, in general, seemed to be very satisfied with the placebos used and the reported median burden to acquire them was relatively low.
      Our study has several limitations: First, the title and abstract screening was not conducted in duplicate. As at that stage, only obviously ineligible studies were excluded, and due to the fact that the screening was used to identify investigators to be invited to a survey, we do not believe that this might have an impact on the overall message of our study. Second, the response rate in the survey was only 26%. Although this is comparable to other online surveys [
      • Hopewell S.
      • Witt C.M.
      • Linde K.
      • Icke K.
      • Adedire O.
      • Kirtley S.
      • et al.
      Influence of peer review on the reporting of primary outcome(s) and statistical analyses of randomised trials.
      ,
      • Danko K.J.
      • Dahabreh I.J.
      • Ivers N.M.
      • Moher D.
      • Grimshaw J.M.
      Contacting authors by telephone increased response proportions compared with emailing: results of a randomized study.
      ] and the sample seems to be representative in terms of measured baseline characteristics, we cannot know if the investigators who responded constitute a self-selected sample experiencing more problems compared to nonresponders. Third, the sample size was relatively small. This is especially the case for questions where only a subsample of investigators was asked or analyzed. Fourth, our search used key elements from the Cochrane Highly Sensitive Search Strategy [], which was adapted to retrieve specifically placebo-controlled trials. It is possible that we missed some potentially relevant RCTs in our search. We do not believe that missing individual RCTs affects the overall meaning of this study. In contrast to clinical systematic reviews where missing a single important RCT can have a major effect on the overall interpretation of the review, the search strategy in our study was preliminarily used to identify corresponding authors in order to invite them to participate in our survey.
      In conclusion, providing trial drugs and matching placebos might be a tool for drug manufacturers to selectively support trials with a hypothesis in their interests. When the drug manufacturer does not provide the placebo, simple solutions such as overencapsulation or using saline seem to be the most popular ways to achieve matching between drugs and placebos. However, in cases where these solutions are not desirable or not possible (e.g., tablet too large to overencapsulate; opening might lead to breaking the blind) it would be important that other sources are available to receive matching placebos. A way forward might be to set up platforms with manufacturers of affordable placebos or to oblige original manufacturers to provide placebos for a reasonable price. Furthermore, investigators should be encouraged to openly report when a placebo does not match the original drug.

      Acknowledgments

      We would like to express our thanks to all participating investigators. In particular we would like to thank Abdulhalim Jamal Kinsara for filling out the questionnaire and for giving us additional feedback.

      References

        • Duley L.
        • Antman K.
        • Arena J.
        • Avezum A.
        • Blumenthal M.
        • Bosch J.
        • et al.
        Specific barriers to the conduct of randomized trials.
        Clin Trials. 2008; 5: 40-48
        • Collins R.
        • MacMahon S.
        Reliable assessment of the effects of treatment on mortality and major morbidity, I: clinical trials.
        Lancet. 2001; 357: 373-380
        • Suvarna V.
        Investigator initiated trials (IITs).
        Perspect Clin Res. 2012; 3: 119-121
        • Konwar M.
        • Bose D.
        • Gogtay N.J.
        • Thatte U.M.
        Investigator-initiated studies: challenges and solutions.
        Perspect Clin Res. 2018; 9: 179-183
        • Speich B.
        • Bausch K.
        • Roth J.A.
        • Hemkens L.G.
        • Ewald H.
        • Vogt D.R.
        • et al.
        Single-dose versus 3-day cotrimoxazole prophylaxis in transurethral resection or greenlight laser vaporisation of the prostate: study protocol for a multicentre randomised placebo controlled non-inferiority trial (CITrUS trial).
        Trials. 2019; 20: 142
        • Han Y.
        • Guo J.
        • Zheng Y.
        • Zang H.
        • Su X.
        • Wang Y.
        • et al.
        Bivalirudin vs heparin with or without tirofiban during primary percutaneous coronary intervention in acute myocardial infarction: the BRIGHT randomized clinical trial.
        JAMA. 2015; 313: 1336-1346
        • Wan M.
        • Orlu-Gul M.
        • Legay H.
        • Tuleu C.
        Blinding in pharmacological trials: the devil is in the details.
        Arch Dis Child. 2013; 98: 656-659
        • Whitham D.
        • Silcocks P.
        • Whitehouse W.
        • Hodgson S.
        • Sammons H.
        An example of problems that arise from clinical trials and how to avoid them.
        Pharm J. 2009; 283: 129-130
        • Curfman G.D.
        • Morrissey S.
        • Drazen J.M.
        Products at risk.
        N Engl J Med. 2010; 363: 1763
        • Christensen M.
        • Knop F.K.
        The unobtainable placebo: control of independent clinical research by industry?.
        Lancet. 2012; 379: 30
        • Martin B.K.
        • Meinert C.L.
        • Breitner J.C.
        • Group A.R.
        Double placebo design in a prevention trial for Alzheimer's disease.
        Control Clin Trials. 2002; 23: 93-99
        • Speich B.
        • Schur N.
        • Gryaznov D.
        • von Niederhausern B.
        • Hemkens L.G.
        • Schandelmaier S.
        • et al.
        Resource use, costs, and approval times for planning and preparing a randomized clinical trial before and after the implementation of the new Swiss human research legislation.
        PLoS One. 2019; 14: e0210669
        • Speich B.
        • von Niederhausern B.
        • Blum C.A.
        • Keiser J.
        • Schur N.
        • Furst T.
        • et al.
        Retrospective assessment of resource use and costs in two investigator-initiated randomized trials exemplified a comprehensive cost item list.
        J Clin Epidemiol. 2018; 96: 73-83
        • Murad M.H.
        • Wang Z.
        Guidelines for reporting meta-epidemiological methodology research.
        Evid Based Med. 2017; 22: 139-142
        • Kelley K.
        • Clark B.
        • Brown V.
        • Sitzia J.
        Good practice in the conduct and reporting of survey research.
        Int J Qual Health Care. 2003; 15: 261-266
        • Webster R.K.
        • Howick J.
        • Hoffmann T.
        • Macdonald H.
        • Collins G.S.
        • Rees J.L.
        • et al.
        Inadequate description of placebo and sham controls in a systematic review of recent trials.
        Eur J Clin Invest. 2019; 49: e13169
        • Hopewell S.
        • Witt C.M.
        • Linde K.
        • Icke K.
        • Adedire O.
        • Kirtley S.
        • et al.
        Influence of peer review on the reporting of primary outcome(s) and statistical analyses of randomised trials.
        Trials. 2018; 19: 30
        • Danko K.J.
        • Dahabreh I.J.
        • Ivers N.M.
        • Moher D.
        • Grimshaw J.M.
        Contacting authors by telephone increased response proportions compared with emailing: results of a randomized study.
        J Clin Epidemiol. 2019; 115: 150-159
        • Cochrane Training
        (Available at:)