Journal of Clinical Epidemiology
Volume 63, Issue 4 , Pages 355-369 , April 2010

Methods to elicit beliefs for Bayesian priors: a systematic review

  • Sindhu R. Johnson

      Affiliations

    • Division of Rheumatology, Department of Medicine, University Health Network, Toronto, Ontario, Canada
    • Department of Health Policy, Management and Evaluation, University of Toronto, Toronto, Ontario, Canada
    • Corresponding Author InformationCorresponding author. Division of Rheumatology, University Health Network, Ground Floor, East Wing, Toronto Western Hospital, 399 Bathurst Street, Toronto, Ontario M5T 2S8, Canada. Tel.: +416-603-6417; fax +416-603-4348.
    • ,
  • George A. Tomlinson

      Affiliations

    • Department of Health Policy, Management and Evaluation, University of Toronto, Toronto, Ontario, Canada
    • Dalla Lana School of Public Health, University of Toronto, Toronto, Ontario, Canada
    • Division of Clinical Decision Making and Health Care, Toronto General Research Institute, Toronto, Ontario, Canada
    • ,
  • Gillian A. Hawker

      Affiliations

    • Department of Health Policy, Management and Evaluation, University of Toronto, Toronto, Ontario, Canada
    • Division of Rheumatology, Department of Medicine, Women's College Hospital, Toronto, Ontario, Canada
    • ,
  • John T. Granton

      Affiliations

    • Divisions of Respirology and Critical Care Medicine, Department of Medicine, University Health Network, Toronto, Ontario, Canada
    • ,
  • Brian M. Feldman

      Affiliations

    • Department of Health Policy, Management and Evaluation, University of Toronto, Toronto, Ontario, Canada
    • Dalla Lana School of Public Health, University of Toronto, Toronto, Ontario, Canada
    • Division of Rheumatology, Department of Paediatrics, The Hospital for Sick Children, Toronto, Ontario, Canada

,Accepted 9 June 2009.

References 

  1. Berry DA. Bayesian clinical trials. Nat Rev Drug Discov. 2006;5:27–36
  2. Spiegelhalter DJ, Abrams KR, Myles JP. An overview of the Bayesian approach. Chichester: John Wiley & Sons Ltd; 2004;Bayesian approaches to clinical trials and health-care evaluation 49–120
  3. Chaloner K, Rhame FS. Quantifying and documenting prior beliefs in clinical trials. Stat Med. 2001;4:581–600
  4. Carlin BP, Chaloner K, Church T, Louis TA, Matts JP. Bayesian approaches for monitoring clinical trials with an application to toxoplasmic encephalitis prophylaxis. Statistician. 1993;42:355–367
  5. White IR, Pocock SJ, Wang D. Eliciting and using expert opinions about influence of patient characteristics on treatment effects: a Bayesian analysis of the CHARM trials. Stat Med. 2005;24:3805–3821
  6. Moye LA. Bayesians in clinical trials: asleep at the switch. Stat Med. 2008;27:469–482
  7. Spiegelhalter DJ. Incorporating Bayesian ideas into health-care evaluation. Stat Sci. 2004;19:156–174
  8. Singh JA, Solomon DH, Dougados M, Felson D, Hawker G, Katz P, et al. Development of classification and response criteria for rheumatic diseases. Arthritis Rheum. 2006;55:348–352
  9. Johnson SR, Hawker GA, Davis AM. The health assessment questionnaire disability index and scleroderma health assessment questionnaire in scleroderma trials: an evaluation of their measurement properties. Arthritis Rheum. 2005;53:256–262
  10. Streiner DL, Norman GR. Health measurement scales. 3rd edition. NewYork: Oxford University Press; 2003;A practical guide to their development and use
  11. Liang MH. Longitudinal construct validity: establishment of clinical meaning in patient evaluative instruments. Med Care. 2000;38(9 Suppl):II84–II90
  12. Feinstein AR. The theory and evaluation of sensibility. In:  Feinstein AR editors. Clinimetrics. New Haven: Yale University Press; 1987;p. 141–165
  13. Errington RD, Ashby D, Gore SM, Abrams KR, Myint S, Bonnett DE, et al. High energy neutron treatment for pelvic cancers: study stopped because of increased mortality. BMJ. 1991;302:1045–1051
  14. Bergus GR, Chapman GB, Gjerde C, Elstein AS. Clinical reasoning about new symptoms despite preexisting disease: sources of error and order effects. Fam Med. 1995;27:314–320
  15. Chaloner K, Church T, Louis TA, Matts JP. Graphical elicitation of a prior distribution for a clinical trial. Statistician. 1993;42:341–353
  16. Freedman LS, Spiegelhalter DJ. The assessment of subjective opinion and its use in relation to stopping rules for clinical trials. Statistician. 1983;32:153–160
  17. Chaloner K. Elicitation of prior distributions. In:  Berry DA,  Stangl DK editor. Bayesian biostatistics. New York: Marcel Dekker Inc; 1996;p. 141–156
  18. de Vet HC, Kessels AG, Leffers P, Knipschild PG. A randomized trial about the perceived informativeness of new empirical evidence. Does beta-carotene prevent (cervical) cancer?. J Clin Epidemiol. 1993;46:509–517
  19. Evans JS, Handley SJ, Over DE, Perham N. Background beliefs in Bayesian inference. Mem Cogn. 2002;2:179–190
  20. Spiegelhalter DJ, Freedman LS, Parmar MK. Applying Bayesian ideas in drug development and clinical trials. Stat Med. 1993;12:1501–1511
  21. Flournoy N. A clinical experiment in bone marrow transplantation: estimating a percentage point of a quantal response curve. In:  Gatsonis C,  Hodges JS,  Kass RE,  Singpurwalla ND editor. Lecture notes in statistics. New York: Sringer-Verlag; 1994;p. 324–335
  22. Garthwaite PH, Dickey JM. An elicitation method for multiple linear regression models. J Behav Decis Making. 1991;4:17–31
  23. Garthwaite PH, Dickey JM. Elicitation of prior distributions for variable selection problems in regression. Ann Stat. 1992;20:1697–1719
  24. Gustafson DH, Sainfort F, Eichler M, Adams L, Bisognano M, Steudel H. Developing and testing a model to predict outcomes of organizational change. Health Serv Res. 2003;38:751–776
  25. Hughes MD. Practical reporting of Bayesian analyses of clinical trials. Drug Inf J. 1991;3:381–393
  26. Spiegelhalter DJ, Freedman LS. A predictive approach to selecting the size of a clinical trial, based on subjective clinical opinion. Stat Med. 1986;5(1):1–13
  27. Hutton JL, Owens RG. Bayesian sample size calculation and prior beliefs about child sexual abuse. Statistician. 1993;42:399–404
  28. Johnson NP, Fisher RA, Braunholtz DA, Gillett WR, Lilford RJ. Survey of Australasian clinicians' prior beliefs concerning lipiodol flushing as a treatment for infertility: a Bayesian study. Aust NZ J Obstet Gyn. 2006;4:298–304
  29. Jones P, Johanson R, Baldwin KJ, Lilford R, Jones P. Changing belief in obstetrics: impact of two multicentre randomised controlled trials. Lancet. 1998;352:1988–1989
  30. Kadane JB. Progress toward a more ethical method for clinical trials. J Med Philos. 1986;11:385–404
  31. Kadane JB. An application of robust Bayesian analysis to a medical experiment. J Stat Plan Infer. 1994;40:221–232
  32. Kadane JB, Wolfson LJ. Experiences in elicitation. Statistician. 1998;47:3–19
  33. Lehmann HP, Goodman SN. Bayesian communication: a clinically significant paradigm for electronic publication. J Am Med Technol. 2000;3:254–266
  34. Li Y, Krantz DH. Experimental tests of subjective Bayesian methods. Psychol Rec. 2005;55:251–277
  35. Lilford R. Formal measurement of clinical uncertainty: prelude to a trial in perinatal medicine. The Fetal Compromise Group. BMJ. 1994;308:111–112
  36. Lilford RJ, Braunholtz D. The statistical basis of public policy: a paradigm shift is overdue. BMJ. 1996;7057:603–607
  37. O'Hagan A. Eliciting expert beliefs in substantial practical application. Statistician. 1998;47:21–35
  38. Parmar MK, Spiegelhalter DJ, Freedman LS. The CHART trials: Bayesian design and monitoring in practice. CHART Steering Committee. Stat Med. 1994;13:1297–1312
  39. Parmar MK, Griffiths GO, Spiegelhalter DJ, Souhami RL, Altman DG, van der SE. Monitoring of large randomised clinical trials: a new approach with Bayesian methods. Lancet. 2001;358:375–381
  40. Ramachandran G. Retrospective exposure assessment using Bayesian methods. Ann Occup Hyg. 2001;45:651–667
  41. Tan S-B, Chung Y-F, Tai B-C, Cheung Y-B, Machin D. Elicitation of prior distributions for a phase III randomized controlled trial of adjuvant therapy with surgery for hepatocellular carcinoma. Control Clin Trials. 2003;2:110–121
  42. Van Der Wilt GJ, Rovers M, Straatman H, Van Der BS, Van Den BP, Zielhuis G. Policy relevance of Bayesian statistics overestimated?. Int J Technol Assess. 2004;4:488–492
  43. Rovers MM, Van Der Wilt GJ, Van Der BS, Straatman H, Ingels K, Zielhuis GA. Bayes' theorem: a negative example of a RCT on grommets in children with glue ear. Eur J Epidemiol. 2005;1:23–28
  44. Winkler RL. The assessment of prior distributions in Bayesian analysis. J Am Stat Assoc. 1967;62:776–800
  45. Van der Fels-Klerx IH, Goossens LH, Saatkamp HW, Horst SH. Elicitation of quantitative data from a heterogeneous expert panel: formal process and application in animal health. Risk Anal. 2002;22:67–81
  46. Carter BL, Butler CD, Rogers JC, Holloway RL. Evaluation of physician decision making with the use of prior probabilities and a decision-analysis model. Arch Fam Med. 1993;2:529–534
  47. Normand SL, Frank RG, McGuire TG. Using elicitation techniques to estimate the value of ambulatory treatments for major depression. Med Decis Making. 2002;22:245–261
  48. Parmar MKB, Ungerleider RS, Simon R. Assessing whether to perform a confirmatory randomized clinical trial. J Natl Cancer I. 1996;22:1645–1651
  49. Winkler RL. Probabilistic prediction: some experimental results. J Am Stat Assoc. 1971;66:675–685
  50. Clemen RT, Wolmark N. Combining probability distributions from experts in risk analysis. Risk Anal. 1999;19:187–203
  51. Murphy AH, Winkler RL. Reliability of subjective probability forecasts of precipitation and temperature. Appl Statist. 1977;26:41–47
  52. Wallsten TS, Budescu DV. Encoding subjective probabilities: a psychological and psychometric review. Manage Sci. 1983;29:151–173
  53. Spiegelhalter DJ, Freedman LS, Parmar MK. Bayesian approaches to randomized trials. J R Statist Soc A. 1994;157:357–416
  54. Hogarth RM. Cognitive processes and the assessment of subjective probability distributions. J Am Stat Assoc. 1975;70:271–294
  55. Evans JS, Brooks P, Pollard P. Prior beliefs and statistical inference. Br J Psychiatry. 1985;76:469–477
  56. Winkler RL. The quantification of judgement: some methodological suggestions. J Am Stat Assoc. 1967;62:1105–1120
  57. Savage LJ. Elicitation of personal probabilities and expectations. J Am Stat Assoc. 1971;66:783–801
  58. Dumouchel W. A Bayesian model and a graphical elicitation procedure for multiple comparisons. Bayesian Stat. 1988;3:127–145
  59. Genest C, Zidek JV. Combining probability distributions: a critique and an annotated bibliography. Stat Sci. 1986;1:114–148
  60. Hawker GA, Gignac MA. How meaningful is our evaluation of meaningful change in osteoarthritis?. J Rheumatol. 2006;33:639–641
  61. O'Hagan A, Buck CE, Daneshkhah A, Eiser JR, Garthwaite PH, Jenkinson DJ, et al. Fundamentals of probability and judgement. Chichester: John Wiley & Sons Ltd; 2006;Uncertain judgements. Eliciting experts' probabilites 1–24
  62. Trochim WM. The research methods knowledge base. 2nd edition.. Cincinnati, Ohio: Atomic Dog Publishing; 2006;
  63. Morgan MG, Henrion M. Human judgement about and with uncertainty. Cambridge: Cambridge University Press; 1990;Uncertainty. A guide to dealing with uncertainty in quantitative risk and policy analysis 102–140
  64. McClish DK, Powell SH. How well can physicians estimate mortality in a medical intensive care unit?. Med Decis Making. 1989;9:125–132
  65. O'Hagan A, Buck CE, Daneshkhah A, Eiser JR, Garthwaite PH, Jenkinson DJ, et al. The psychology of judgement under uncertainty. Chichester: John Wiley & Sons Ltd; 2006;Uncertain judgements. Eliciting experts' probabilites 33–60
  66. Hiance A, Chevret S, Levy V. A practical approach for eliciting expert prior beliefs about cancer survival in phase III randomized trial. J Clin Epidemiol. 2009;62:431–437
  67. Abrams K, Ashby D, Errington D. Simple Bayesian analysis in clinical trials: a tutorial. Control Clin Trials. 1994;5:349–359
  68. Kadane JB, Dickey JM, Winkler RL, Smith WS, Peters SC. Interactive elicitation of opinion for a normal linear model. J Am Stat Assoc. 1980;75:845–854
  69. Kadane JB. Subjective Bayesian analysis for surveys with missing data. Statistician. 1992;42:415–426
  70. Ten Centre Study Group. Ten centre trial of artificial surfactant (artificial lung expanding compound) in very premature babies. Br Med J (Clin Res Ed). 1987;294:991–996

PII: S0895-4356(09)00175-9

doi: 10.1016/j.jclinepi.2009.06.003

Journal of Clinical Epidemiology
Volume 63, Issue 4 , Pages 355-369 , April 2010

Article Tools

* Image Usage & Resolution