Journal of Clinical Epidemiology
Volume 59, Issue 11 , Pages 1174-1182 , November 2006

Computer adaptive testing improved accuracy and precision of scores over random item selection in a physical functioning item bank

  • Stephen M. Haley

      Affiliations

    • Department of Rehabilitation Sciences, Sargent College of Health and Rehabilitation Sciences, Boston University, 635 Commanwealth Ave, Boston, MA 02215, USA
    • Health and Disability Research Institute, Boston University, 53 Bay State Road, Boston, MA 02215, USA
    • Corresponding Author InformationCorresponding author. Tel.: (617) 353-1295; fax: (617) 358-1355.
    • ,
  • Pengsheng Ni

      Affiliations

    • Health and Disability Research Institute, Boston University, 53 Bay State Road, Boston, MA 02215, USA
    • ,
  • Ronald K. Hambleton

      Affiliations

    • Center for Educational Assessment, Department of Educational Policy, Research and Administration, School of Education, University of Massachusetts Amherst, Hills South-Room 154, Amberst, MA 01003, USA
    • ,
  • Mary D. Slavin

      Affiliations

    • Health and Disability Research Institute, Boston University, 53 Bay State Road, Boston, MA 02215, USA
    • ,
  • Alan M. Jette

      Affiliations

    • Department of Rehabilitation Sciences, Sargent College of Health and Rehabilitation Sciences, Boston University, 635 Commanwealth Ave, Boston, MA 02215, USA

,Accepted 27 February 2006.

References 

  1. Kane RL. Looking for physical therapy outcomes. Phys Ther. 1994;74:425–429
  2. Jette AM. Measuring subjective clinical outcomes. Phys Ther. 1989;69:580–584
  3. Latham NK, Haley SM. Measuring functional outcomes across postacute care: current challenges and future directions. Crit Rev Phys Rehabil Med. 2003;15:83–98
  4. Jette A, Haley S. Contemporary measurement techniques for rehabilitation outcome assessment. J Rehabil Med, in press.
  5. Hays RD, Morales LS, Reise SP. Item response theory and health outcomes measurement in the 21st century. Med Care. 2000;38:II-28–II-42
  6. Ware JE, Bjorner JB, Kosinski M. Practical implications of item response theory and computerized adaptive testing: a brief summary of ongoing studies of widely used Headache Impact Scales. Med Care. 2000;38:II-73–II-82
  7. Andresen EM. Criteria for assessing the tools of disability outcomes research. Arch Phys Med Rehabil. 2000;81:S15–S20
  8. Jette AM, Haley SM, Ni PS. Comparison of functional status tools used in post-acute care. Health Care Finance Rev. 2003;24:13–24
  9. Revicki DA, Cella DF. Health status assessment for the twenty-first century: item response theory, item banking and computer adaptive testing. Qual Life Res. 1997;6:595–600
  10. Hambleton R, Zaal J, Pieters J. Computerized adaptive testing: theory, applications, and standards. In:  Hambleton R,  Zaal J editor. Advances in educational and psychological testing: theory and applications. Boston: Kluwer Academic Publishers; 1991;p. 341–366
  11. Luecht R. Computer-adaptive testing. In:  Everett B,  Howell D editor. Encyclopedia of statistics in behavioral science. New York: Wiley; 2004;
  12. Wainer H. Computerized adaptive testing: a primer. Mahwah, NJ: Lawrence Erlbaum Associates; 2000;
  13. Veldkamp B, van der Linden W. Designing item pools for computerized adaptive testing. In:  van der Linden W,  Glas C editor. Computerized adaptive testing: theory and practice. Dordrecht, The Netherlands: Kluwer Academic Publishers; 2000;p. 149–162
  14. Mills CN, Steffen M. The GRE computer adaptive test: operational issues. In:  van der Linden WJ,  Glas CAW editor. Computerized adaptive testing: theory and practice. Dordrecht, The Netherlands: Kluwer Academic Publishers; 2000;p. 75–99
  15. van der Linden WJ. The changing conception of measurement in education and psychology. Appl Psychol Measur. 1986;10:325–332
  16. Haley SM, Raczek AE, Coster WJ, Dumas HM, Fragala-Pinkham MA. Assessing mobility in children using a computer adaptive testing version of the Pediatric Evaluation of Disability Inventory (PEDI). Arch Phys Med Rehabil. 2005;86:932–939
  17. Haley SM, Coster WJ, Andres PL, Kosinski M, Ni PS. Score comparability of short-forms and computerized adaptive testing: simulation study with the Activity Measure for Post-Acute Care (AM-PAC). Arch Phys Med Rehabil. 2004;85:661–666
  18. Gardner W, Kelleher KJ, Pajer KA. Multidimensional adaptive testing for metal health problems in primary care. Med Care. 2002;40:812–823
  19. Dijkers MP. A computer adaptive testing simulation applied to the FIM instrument motor component. Arch Phys Med Rehabil. 2003;84:384–393
  20. Ware J, Sinclair S, Gandek B, Bjorner B. Item response theory in computer adaptive testing: implications for outcomes measurement in rehabilitation. Rehabil Psychol. 2005;50:71–78
  21. Ware JE, Kosinski M, Bjorner JB, et al. Applications of computerized adaptive testing (CAT) to the assessment of headache impact. Qual Life Res. 2003;12:935–952
  22. Lai J-S, Cella D, Chang C-H, Bode RK, Heinemann AW. Item banking to improve, shorten and computerize self-reported fatigue: an illustration of steps to create a core item bank from the FACIT-Fatigue Scale. Qual Life Res. 2003;12:485–501
  23. Cella D, Chang C-H. A discussion of item response theory and its applications in health status assessment. Med Care. 2000;38:66–72
  24. Hambleton RK. Principles and selected applications of item response theory. In:  Linn RL editors. Educational measurement. 3rd ed.. Phoenix, AZ: The Oryx Press; 1993;p. 147–200
  25. World Health Organization . International Classification of Functioning, Disability and Handicap (ICF). Geneva: World Health Organization; 2001;
  26. Stucki G, Cieza A, Ewert T, Kostanjsek N, Chatterji S, Ustun T. Application of the International Classification of Functioning, Disability and Health (ICF) in clinical practice. Disabil Rehabil. 2002;24:281–282
  27. Stucki G, Ewert T, Cieza A. Value and application of the ICF in rehabilitation medicine. Disabil Rehabil. 2003;25:628–634
  28. Haley SM, Andres PL, Coster WJ, Kosinski M, Ni PS, Jette A. Short-form activity measure for post-acute care (AM-PAC). Arch Phys Med Rehabil. 2004;85:649–660
  29. Haley SM, Coster WJ, Andres PL, et al. Activity outcome measurement for post-acute care. Med Care. 2004;42:I-49–I-61
  30. Ware J, Kosinski M, Dewey J, Gandek B. How to score and interpret single-item health status measures: a manual for users of the SF-8™ Health Survey. Lincoln, RI: QualityMetric; 1999;
  31. de Haan R, Horn J, Limburg M, Van Der Meulen J, Bossuyt P. A comparison of five stroke scales with measures of disability, handicap, and quality of life. Stroke. 1993;24:1178–1181
  32. Coster WJ, Haley SM, Andres PL, Ludlow LH, Bond TLY, Ni PS. Refining the conceptual basis for rehabilitation outcome measurement: personal care and instrumental activities domain. Med Care. 2004;42:I-62–I-72
  33. Coster WJ, Haley SM, Ludlow LH, Andres PL, Ni PS. Development of an applied cognition scale to measure rehabilitation outcomes. Arch Phys Med Rehabil. 2004;85:2030–2035
  34. Ware J, Kosinski M. The SF-36® Physical and Mental Health Summary Scales: a manual for user's of version 1.0. 2nd ed.. Boston, MA: Health Assessment Lab; 2001;
  35. Haley SM, McHorney CA, Ware JE. Evaluation of the MOS SF-36 Physical Functioning Scale (PF-10): I. Unidimensionality and reproducibility of the Rasch item scale. J Clin Epidemiol. 1994;47:671–684
  36. Andres PL, Haley SM, Ni PS. Is patient-reported function reliable for monitoring post-acute outcomes?. Am J Phys Med Rehabil. 2003;82:614–621
  37. Zumbo B. A handbook on the theory and methods of Differential Item Functioning (DIF). Ottawa, CA: Directorate of Human Resources Research and Evaluation; 1999;
  38. Jodoin MG, Gierl MJ. Evaluating type I error and power rates using an effect size measure with the logistic regression procedure for DIF detection. Appl Measur Ed. 2001;4:329–349
  39. Muthen B, Muthen L. Mplus user's guide. Los Angeles: Muthen & Muthen; 2001;
  40. Lord FM. Applications of item response theory to practical testing problems. Hillsdale, NJ: Lawrence Erlbaum Associates; 1980;
  41. Hambleton R, Swaminathan H, Rogers H. Fundamentals of item response theory. Newbury Park, CA: Sage Publications; 1991;
  42. Mislevy RJ. Recent developments in the factor analysis of categorical variables. J Ed Stat. 1986;11:3–31
  43. Muthen BO, Muthen L. Mplus user's guide. Los Angeles, CA: Muthen & Muthen; 1988;
  44. Yen WM. Scaling performance assessments: strategies for managing local item dependence. J Ed Measur. 1993;30:187–213
  45. Muraki E. A generalized partial credit model. In:  van der Linden W,  Hambleton R editor. Handbook of modern item response theory. New York: Springer-Verlag; 1997;p. 153–168
  46. Masters G, Wright B. The partial credit model. In:  van der Linden W,  Hambleton R editor. Handbook of modern item response theory. New York: Springer-Verlag; 1997;p. 101–121
  47. Muraki E, Bock RD. PARSCALE: IRT Item analysis and test scoring for rating scale data. Chicago: Scientific Software International; 1997;
  48. Drasgrow F, Levine MV, Tsien S, Williams B, Mead AD. Fitting polytomous item response theory models to multiple-choice tests. Appl Psychol Measur. 1995;19:146–165
  49. McBride J. Research antecedents of applied adaptive testing. In:  Sands W,  Waters B,  McBride J editor. Computer adaptive testing: from inquiry to operation. Washington, DC: American Psychological Association; 1997;
  50. Crosby RD, Kolotkin RL, Williams GR. Defining clinically meaningful change in health-related quality of life. J Clin Epidemiol. 2003;56:395–407
  51. Fries JF, Bruce B, Cella D. The promise of PROMIS: Using item response theory to improve assessment of patient-reported outcomes. Clin Exp Rheumatol. 2005;23(Suppl. 39):S53–S57
  52. Yuan K-H. Fit indices versus test statistics. Multilevel Behav Res. 2005;40:115–148
  53. Hu LT, Bentler PM. Evaluating model fit. In:  Hoyle RH editors. Structural equation modeling. Thousand Oaks, CA: Sage; 1995;p. 76–99
  54. Haley S, Raczek A, Coster W, Dumas H, Fragala-Pinkham M. Assessing mobility in children using a computer adaptive testing version of the Pediatric Evaluation of Disability Inventory (PEDI). Arch Phys Med Rehabil. 2005;86:932–939

PII: S0895-4356(06)00112-0

doi: 10.1016/j.jclinepi.2006.02.010

Journal of Clinical Epidemiology
Volume 59, Issue 11 , Pages 1174-1182 , November 2006

Article Tools

* Image Usage & Resolution