Abstract
Objective
Bayesian inference has the advantage of formally incorporating prior beliefs about
the effect of an intervention into analyses of treatment effect through the use of
prior probability distributions or “priors.” Multiple methods to elicit beliefs from
experts for inclusion in a Bayesian study have been used; however, the measurement
properties of these methods have been infrequently evaluated. The objectives of this
study were to evaluate the feasibility, validity, and reliability of a belief elicitation
method for Bayesian priors.
Study Design and Setting
A single-center, cross-sectional study using a sample of academic specialists who
treat pulmonary hypertension patients was conducted to test the feasibility, face
and construct validity, and reliability of a belief elicitation method. Using this
method, participants expressed the probability of 3-year survival with and without
warfarin. Applying adhesive dots or “chips,” each representing 5% probability, in
“bins” on a line, participants expressed their uncertainty and weight of belief about
the effect of warfarin on 3-year survival.
Results
Of the 12 participants, 11 (92%) reported that the belief elicitation method had face
validity, 10 (83%) found the questions clear, and 11 (92%) found the response option
easy to use. The median time to completion was 10 minutes (5–15 minutes). Internal
validity testing found moderate agreement (weighted kappa=0.54–0.57). The intraclass correlation coefficient for test–retest reliability was
0.93.
Conclusion
This method of belief elicitation for Bayesian priors is feasible, valid, and reliable.
It can be considered for application in Bayesian clinical studies.
Keywords
To read this article in full you will need to make a payment
Purchase one-time access:
Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online accessOne-time access price info
- For academic or personal research use, select 'Academic and Personal'
- For corporate R&D use, select 'Corporate R&D Professionals'
Subscribe:
Subscribe to Journal of Clinical EpidemiologyAlready a print subscriber? Claim online access
Already an online subscriber? Sign in
Register: Create an account
Institutional Access: Sign in to ScienceDirect
References
- Bayesian clinical trials.Nat Rev Drug Discov. 2006; 5: 27-36
- Quantifying and documenting prior beliefs in clinical trials.Stat Med. 2001; 20: 581-600
Johnson SR, Tomlinson GA, Hawker GA, Granton JT, Feldman BM. Methods to elicit beliefs for Bayesian priors. A systematic review. J Clin Epidemiol. In press.
- Bayesians in clinical trials: asleep at the switch.Stat Med. 2008; 27: 469-482
- Practical reporting of Bayesian analyses of clinical trials.Drug Inf J. 1991; 25: 381-393
- The CHART trials: Bayesian design and monitoring in practice. CHART Steering Committee.Stat Med. 1994; 13: 1297-1312
- Assessing whether to perform a confirmatory randomized clinical trial.J Natl Cancer Inst. 1996; 88: 1645-1651
- Elicitation of prior distributions for a phase III randomized controlled trial of adjuvant therapy with surgery for hepatocellular carcinoma.Control Clin Trials. 2003; 24: 110-121
- Methods to elicit beliefs for Bayesian priors: a systematic review.J Clin Epidemiol. 2010; 63: 355-369
- Cognitive processes and the assessment of subjective probability distributions.J Am Stat Assoc. 1975; 70: 271-294
- Encoding subjective probabilities: a psychological and psychometric review.Manage Sci. 1983; 29: 151-173
- Reliability of subjective probability forecasts of precipitation and temperature.Appl Statist. 1977; 26: 41-47
- Clinical reasoning about new symptoms despite preexisting disease: sources of error and order effects.Fam Med. 1995; 27: 314-320
- Prior beliefs and statistical inference.Brit J Psychiatry. 1985; 76: 469-477
- Background beliefs in Bayesian inference.Mem Cognit. 2002; 30: 179-190
- Eliciting and using expert opinions about influence of patient characteristics on treatment effects: a Bayesian analysis of the CHARM trials.Stat Med. 2005; 24: 3805-3821
- The assessment of prior distributions in Bayesian analysis.J Am Stat Assoc. 1967; 62: 776-800
- Elicitation of quantitative data from a heterogeneous expert panel: formal process and application in animal health.Risk Anal. 2002; 22: 67-81
- Experimental tests of subjective Bayesian methods.Psychol Rec. 2005; 55: 251-277
- Elicitation of prior distributions.in: Berry D.A. Stangl D.K. Bayesian biostatistics. Marcel Dekker Inc, New York1996: 141-156
- Eliciting expert beliefs in substantial practical application.Statistician. 1998; 47: 21-35
- The quantification of judgement: some methodological suggestions.J Am Stat Assoc. 1967; 62: 1105-1120
- Using elicitation techniques to estimate the value of ambulatory treatments for major depression.Med Decis Making. 2002; 22: 245-261
- Thrombotic arteriopathy and anticoagulation in pulmonary hypertension.Chest. 2006; 130: 545-552
- Pulmonary hypertension in systemic sclerosis: an analysis of 17 patients.Br J Rheumatol. 1996; 35: 989-993
- Survival in patients with primary pulmonary hypertension. Results from a national prospective registry.Ann Intern Med. 1991; 115: 343-349
- Anticoagulation in pulmonary arterial hypertension: a qualitative systematic review.Eur Respir J. 2006; 28: 999-1004
- For and against: clinical equipoise and not the uncertainty principle is the moral underpinning of the randomised controlled trial.BMJ. 2000; 321: 756-758
- The research methods knowledge base.Second edition. Atomic Dog Publishing, Cincinnati, Ohio2006
- Internet, mail, and mixed-mode surveys. The tailored design method.3rd edition. John Wiley & Sons, Inc, Hoboken, NJ2009
- Health measurement scales. A practical guide to their development and use.3rd edition. Oxford University Press, New York2003
- A new procedure for assessing reliability of scoring EEG sleep recordings.Am J EEG Technol. 1971; 11: 101-109
- Sample size and optimal designs for reliability studies.Stat Med. 1998; 17: 101-110
- Intraclass correlations: uses in assessing rater reliability.Psychol Bull. 1979; 86: 420-428
- Uncertain judgements. Eliciting experts' probabilities.John Wiley & Sons Ltd, Chichester, UK2006
- The health assessment questionnaire disability index and scleroderma health assessment questionnaire in scleroderma trials: an evaluation of their measurement properties.Arthritis Rheum. 2005; 53: 256-262
- Individual-patient monitoring in clinical practice: are available health status surveys adequate?.Qual Life Res. 1995; 4: 293-307
- The measurement of observer agreement for categorical data.Biometrics. 1977; 33: 159-174
- Combining probability distributions from experts in risk analysis.Risk Anal. 1999; 19: 187-203
- Psychometric theory.McGraw-Hill, New York1978
- Longitudinal construct validity: establishment of clinical meaning in patient evaluative instruments.Med Care. 2000; 38: II84-II90
- The effect of anticoagulant therapy in primary and anorectic drug-induced pulmonary hypertension.Chest. 1997; 112: 714-721
- The effect of high doses of calcium-channel blockers on survival in primary pulmonary hypertension.N Engl J Med. 1992; 327: 76-81
- Primary pulmonary hypertension: natural history and the importance of thrombosis.Circulation. 1984; 70: 580-587
- Effects of a combination therapy of anticoagulant and vasodilator on the long-term prognosis of primary pulmonary hypertension.Jpn Circ J. 1993; 57: 63-69
Article info
Publication history
Published online: November 19, 2009
Accepted:
August 6,
2009
Identification
Copyright
© 2010 Elsevier Inc. Published by Elsevier Inc. All rights reserved.
