Journal of Clinical Epidemiology
Volume 63, Issue 4 , Pages 370-383 , April 2010

A valid and reliable belief elicitation method for Bayesian priors

  • Sindhu R. Johnson

      Affiliations

    • Division of Rheumatology, Department of Medicine, Toronto Western Hospital, University Health Network, Toronto, Ontario, Canada
    • Department of Health Policy, Management and Evaluation, University of Toronto, Toronto, Ontario, Canada
    • Corresponding Author InformationCorresponding author. Division of Rheumatology, Ground Floor, East Wing, Toronto Western Hospital, 399 Bathurst Street, Toronto, Ontario M5T 2S8, Canada. Tel.: +416-603-6417; fax: +416-603-4348.
    • ,
  • George A. Tomlinson

      Affiliations

    • Department of Health Policy, Management and Evaluation, University of Toronto, Toronto, Ontario, Canada
    • Dalla Lana School of Public Health, University of Toronto, Toronto, Ontario, Canada
    • Division of Clinical Decision Making and Health Care, Toronto General Research Institute, Toronto, Ontario, Canada
    • ,
  • Gillian A. Hawker

      Affiliations

    • Department of Health Policy, Management and Evaluation, University of Toronto, Toronto, Ontario, Canada
    • Division of Rheumatology, Department of Medicine, Women's College Hospital, Toronto, Ontario, Canada
    • ,
  • John T. Granton

      Affiliations

    • Divisions of Respirology and Critical Care Medicine, Department of Medicine, Toronto General Hospital, Toronto, Ontario, Canada
    • ,
  • Haddas A. Grosbein

      Affiliations

    • Division of Rheumatology, Department of Pediatrics, The Hospital for Sick Children, Toronto, Ontario, Canada
    • ,
  • Brian M. Feldman

      Affiliations

    • Department of Health Policy, Management and Evaluation, University of Toronto, Toronto, Ontario, Canada
    • Dalla Lana School of Public Health, University of Toronto, Toronto, Ontario, Canada
    • Division of Rheumatology, Department of Pediatrics, The Hospital for Sick Children, Toronto, Ontario, Canada

,Accepted 6 August 2009.

References 

  1. Berry DA. Bayesian clinical trials. Nat Rev Drug Discov. 2006;5:27–36
  2. Chaloner K, Rhame FS. Quantifying and documenting prior beliefs in clinical trials. Stat Med. 2001;20:581–600
  3. Johnson SR, Tomlinson GA, Hawker GA, Granton JT, Feldman BM. Methods to elicit beliefs for Bayesian priors. A systematic review. J Clin Epidemiol. In press.
  4. Moye LA. Bayesians in clinical trials: asleep at the switch. Stat Med. 2008;27:469–482
  5. Hughes MD. Practical reporting of Bayesian analyses of clinical trials. Drug Inf J. 1991;25:381–393
  6. Parmar MK, Spiegelhalter DJ, Freedman LS. The CHART trials: Bayesian design and monitoring in practice. CHART Steering Committee. Stat Med. 1994;13:1297–1312
  7. Parmar MKB, Ungerleider RS, Simon R. Assessing whether to perform a confirmatory randomized clinical trial. J Natl Cancer Inst. 1996;88:1645–1651
  8. Tan S-B, Chung Y-F, Tai B-C, Cheung Y-B, Machin D. Elicitation of prior distributions for a phase III randomized controlled trial of adjuvant therapy with surgery for hepatocellular carcinoma. Control Clin Trials. 2003;24:110–121
  9. Johnson SR, Tomlinson GA, Hawker GA, Granton JT, Feldman BM. Methods to elicit beliefs for Bayesian priors: a systematic review. J Clin Epidemiol. 2010;63:355–369
  10. Hogarth RM. Cognitive processes and the assessment of subjective probability distributions. J Am Stat Assoc. 1975;70:271–294
  11. Wallsten TS, Budescu DV. Encoding subjective probabilities: a psychological and psychometric review. Manage Sci. 1983;29:151–173
  12. Murphy AH, Winkler RL. Reliability of subjective probability forecasts of precipitation and temperature. Appl Statist. 1977;26:41–47
  13. Bergus GR, Chapman GB, Gjerde C, Elstein AS. Clinical reasoning about new symptoms despite preexisting disease: sources of error and order effects. Fam Med. 1995;27:314–320
  14. Evans JS, Brooks P, Pollard P. Prior beliefs and statistical inference. Brit J Psychiatry. 1985;76:469–477
  15. Evans JS, Handley SJ, Over DE, Perham N. Background beliefs in Bayesian inference. Mem Cognit. 2002;30:179–190
  16. White IR, Pocock SJ, Wang D. Eliciting and using expert opinions about influence of patient characteristics on treatment effects: a Bayesian analysis of the CHARM trials. Stat Med. 2005;24:3805–3821
  17. Winkler RL. The assessment of prior distributions in Bayesian analysis. J Am Stat Assoc. 1967;62:776–800
  18. Van der Fels-Klerx IH, Goossens LH, Saatkamp HW, Horst SH. Elicitation of quantitative data from a heterogeneous expert panel: formal process and application in animal health. Risk Anal. 2002;22:67–81
  19. Li Y, Krantz DH. Experimental tests of subjective Bayesian methods. Psychol Rec. 2005;55:251–277
  20. Chaloner K. Elicitation of prior distributions. In:  Berry DA,  Stangl DK editor. Bayesian biostatistics. New York: Marcel Dekker Inc; 1996;p. 141–156
  21. O'Hagan A. Eliciting expert beliefs in substantial practical application. Statistician. 1998;47:21–35
  22. Winkler RL. The quantification of judgement: some methodological suggestions. J Am Stat Assoc. 1967;62:1105–1120
  23. Normand SL, Frank RG, McGuire TG. Using elicitation techniques to estimate the value of ambulatory treatments for major depression. Med Decis Making. 2002;22:245–261
  24. Johnson SR, Granton JT, Mehta S. Thrombotic arteriopathy and anticoagulation in pulmonary hypertension. Chest. 2006;130:545–552
  25. Koh ET, Lee P, Gladman DD, Abu-Shakra M. Pulmonary hypertension in systemic sclerosis: an analysis of 17 patients. Br J Rheumatol. 1996;35:989–993
  26. D'Alonzo GE, Barst RJ, Ayres SM, Bergofsky EH, Brundage BH, Detre KM, et al. Survival in patients with primary pulmonary hypertension. Results from a national prospective registry. Ann Intern Med. 1991;115:343–349
  27. Johnson SR, Mehta S, Granton JT. Anticoagulation in pulmonary arterial hypertension: a qualitative systematic review. Eur Respir J. 2006;28:999–1004
  28. Weijer C, Shapiro SH, Cranley GK. For and against: clinical equipoise and not the uncertainty principle is the moral underpinning of the randomised controlled trial. BMJ. 2000;321:756–758
  29. Trochim WM. The research methods knowledge base. Second edition. Cincinnati, Ohio: Atomic Dog Publishing; 2006;
  30. Dillman DS, Smyth JD, Christian LM. Internet, mail, and mixed-mode surveys. The tailored design method. 3rd edition. Hoboken, NJ: John Wiley & Sons, Inc; 2009;
  31. Streiner DL, Norman GR. Health measurement scales. A practical guide to their development and use. 3rd edition. New York: Oxford University Press; 2003;
  32. Cicchetti DV, Allison T. A new procedure for assessing reliability of scoring EEG sleep recordings. Am J EEG Technol. 1971;11:101–109
  33. Walter SD, Eliasziw M, Donner A. Sample size and optimal designs for reliability studies. Stat Med. 1998;17:101–110
  34. Shrout PE, Fleiss JL. Intraclass correlations: uses in assessing rater reliability. Psychol Bull. 1979;86:420–428
  35. O'Hagan A, Buck CE, Daneshkhah A, Eiser JR, Garthwaite PH, Jenkinson DJ, et al. Uncertain judgements. Eliciting experts' probabilities. Chichester, UK: John Wiley & Sons Ltd; 2006;
  36. Johnson SR, Hawker GA, Davis AM. The health assessment questionnaire disability index and scleroderma health assessment questionnaire in scleroderma trials: an evaluation of their measurement properties. Arthritis Rheum. 2005;53:256–262
  37. McHorney CA, Tarlov AR. Individual-patient monitoring in clinical practice: are available health status surveys adequate?. Qual Life Res. 1995;4:293–307
  38. Landis J, Koch G. The measurement of observer agreement for categorical data. Biometrics. 1977;33:159–174
  39. Clemen RT, Wolmark N. Combining probability distributions from experts in risk analysis. Risk Anal. 1999;19:187–203
  40. Nunnally JC. Psychometric theory. New York: McGraw-Hill; 1978;
  41. Liang MH. Longitudinal construct validity: establishment of clinical meaning in patient evaluative instruments. Med Care. 2000;38(9 Suppl):II84–II90
  42. Frank H, Mlczoch J, Huber K, Schuster E, Gurtner HP, Kneussl M. The effect of anticoagulant therapy in primary and anorectic drug-induced pulmonary hypertension. Chest. 1997;112:714–721
  43. Rich S, Kaufmann E, Levy PS. The effect of high doses of calcium-channel blockers on survival in primary pulmonary hypertension. N Engl J Med. 1992;327:76–81
  44. Fuster V, Steele PM, Edwards WD, Gersh BJ, McGoon MD, Frye RL. Primary pulmonary hypertension: natural history and the importance of thrombosis. Circulation. 1984;70:580–587
  45. Ogata M, Ohe M, Shirato K, Takishima T. Effects of a combination therapy of anticoagulant and vasodilator on the long-term prognosis of primary pulmonary hypertension. Jpn Circ J. 1993;57:63–69

PII: S0895-4356(09)00233-9

doi: 10.1016/j.jclinepi.2009.08.005

Journal of Clinical Epidemiology
Volume 63, Issue 4 , Pages 370-383 , April 2010

Article Tools

* Image Usage & Resolution