Journal of Clinical Epidemiology
Volume 64, Issue 3 , Pages 264-269 , March 2011

The limits of agreement and the intraclass correlation coefficient may be inconsistent in the interpretation of agreement

  • Cristina Costa-Santos

      Affiliations

    • Biostatistics and Medical Informatics Department, Faculty of Medicine, University of Porto, Porto, Portugal
    • Center for Research in Health Technologies and Information Systems, Faculty of Medicine, University of Porto, Porto, Portugal
    • Corresponding Author InformationCorresponding author. Biostatistics and Medical Informatics Department, Faculty of Medicine, University of Porto, Al. Prof. Hernâni Monteiro, 4200-319 Porto, Portugal. Tel.: +351-22-551-3622; fax: +351-22-551-3623.
    • ,
  • João Bernardes

      Affiliations

    • Department of Obstetrics and Gynaecology, Faculty of Medicine, University of Porto, Porto, Portugal
    • Instituto de Engenharia Biomédica, Porto, Portugal
    • Department of Obstetrics and Gynaecology, Hospital de S. João, Porto, Portugal
    • ,
  • Diogo Ayres-de-Campos

      Affiliations

    • Department of Obstetrics and Gynaecology, Faculty of Medicine, University of Porto, Porto, Portugal
    • Instituto de Engenharia Biomédica, Porto, Portugal
    • Department of Obstetrics and Gynaecology, Hospital de S. João, Porto, Portugal
    • ,
  • Antónia Costa

      Affiliations

    • Department of Obstetrics and Gynaecology, Faculty of Medicine, University of Porto, Porto, Portugal
    • Instituto de Engenharia Biomédica, Porto, Portugal
    • Department of Obstetrics and Gynaecology, Hospital de S. João, Porto, Portugal
    • ,
  • Célia Costa

      Affiliations

    • Department of Obstetrics and Gynaecology, Faculty of Medicine, University of Porto, Porto, Portugal
    • Instituto de Engenharia Biomédica, Porto, Portugal

,Accepted 20 November 2009.

References 

  1. de Vet H. Observer reliability and agreement. In:  Armitage P,  Colton T editor. Encyclopedia of biostatistics. 1st edition. Chichester, UK: Wiley; 1999;p. 3123–3127
  2. Ayres-de-Campos D, Bernardes J, Costa-Pereira A, Pereira-Leite L. Inconsistencies in expert's classification of cardiotocograms and subsequent clinical decision. Br J Obstet Gynaecol. 1999;106:1307–1310
  3. Santos CC, Pereira CA, Bernardes J. Agreement studies in obstetrics and gynaecology: inappropriateness, controversies and consequences. Br J Obstet Gynaecol. 2005;112:667–669
  4. Luiz RR, Szklo M. More than one statistical strategy to assess agreement of quantitative measurements may usefully be reported. J Clin Epidemiol. 2003;56:963–967
  5. Bland JM, Altman DG. Statistical methods for assessing two methods of clinical measurement. Lancet. 1986;1:307–310
  6. Bartko JJ. The intraclass correlation coefficient as a measure of reliability. Psychol Rep. 1966;19:3–11
  7. Muller R, Buttner P. A critical discussion of intraclass correlation coefficients. Stat Med. 1994;13:2465–2476
  8. Burdock E, Fleiss JL, Hardesty AS. A new view of interobserver agreement. Person Psychol. 1963;16:373–384
  9. Casey BM, Mclntire DD, Leveno KJ. The continuing value of the Apgar score for the assessment of newborn infants. N Engl J Med. 2001;344:467–471
  10. van de Riet JE, Vandenbussche FP, Le Cessie S, Keirse MJ. Newborn assessment and long-term adverse outcome: a systematic review. Am J Obstet Gynecol. 1999;180:1024–1029
  11. Shrout PE, Fleiss JL. Intraclass correlations: uses in assessing rater reliability. Psychol Bull. 1979;86:420–427
  12. Fleiss JU, Davies M. Measuring nominal scale agreement among many raters. Psychol Bull. 1971;76:378–382
  13. Carpenter J, Bithell J. Bootstrap confidence intervals: when, which, what? A practical guide for medical statisticians. Stat Med. 2000;19:1141–1164
  14. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33:159–174

PII: S0895-4356(09)00364-3

doi: 10.1016/j.jclinepi.2009.11.010

Journal of Clinical Epidemiology
Volume 64, Issue 3 , Pages 264-269 , March 2011

Article Tools