Journal of Clinical Epidemiology
Volume 61, Issue 11 , Pages 1113-1124 , November 2008

Computerized adaptive test for patients with knee impairments produced valid and responsive measures of function

  • Dennis L. Hart

      Affiliations

    • Focus On Therapeutic Outcomes, Inc., 551 Yopps Cove Road, White Stone, VA, 22578-2403, USA
    • Corresponding Author InformationCorresponding author. Consulting and Research, Focus On Therapeutic Outcomes, Inc., 551 Yopps Cove Road, White Stone, VA 22578-2403, USA. Tel.: +804-436-9727.
    • ,
  • Ying-Chih Wang

      Affiliations

    • Focus On Therapeutic Outcomes, Inc., 551 Yopps Cove Road, White Stone, VA, 22578-2403, USA
    • ,
  • Paul W. Stratford

      Affiliations

    • School of Rehabilitation Science and Department of Clinical Epidemiology and Biostatistics, McMaster University, Ontario, Canada
    • ,
  • Jerome E. Mioduski

      Affiliations

    • Focus On Therapeutic Outcomes, Inc., 551 Yopps Cove Road, White Stone, VA, 22578-2403, USA

,Accepted 20 January 2008.

References 

  1. Fliege H, Becker J, Walter OB, Bjorner JB, Klapp BF, Rose M. Development of a computer-adaptive test for depression (D-CAT). Qual Life Res. 2005;14:2277–2291
  2. Kosinski M, Bjorner JB, Ware JE, Sullivan E, Straus WL. An evaluation of a patient-reported outcomes found computerized adaptive testing was efficient in assessing osteoarthritis impact. J Clin Epidemiol. 2006;59:715–723
  3. Ware JE, Bjorner JB, Kosinski M. Practical implications of item response theory and computerized adaptive testing: a brief summary of ongoing studies of widely used headache impact scales. Med Care. 2000;38(9 Suppl):II73–II82
  4. In:  Mills CN,  Potenza MT,  Fremer JJ,  Ward WC editor. Computer-based testing. Building the foundation for future assessments. Mahwah, NJ: Lawrence Erlbaum Associates; 2002;
  5. In:  Sands WA,  Waters BK,  McBride JR editor. Computerized adaptive testingFrom inquiry to operation. Washington, DC: American Psychological Association; 1997;
  6. In:  Wainer H editors. Computerized adaptive testing. A primer. 2nd edition. Mahwah, NJ: Lawrence Erlbaum Associates; 2000;
  7. Hambleton RK, Swaminathan H, Rogers HJ. Fundamentals of item response theory. Newbury Park, CA: Sage; 1991;
  8. Hays RD, Morales LS, Reise SP. Item response theory and health outcomes measurement in the 21st century. Med Care. 2000;38(9 Suppl):II28–II42
  9. Bjorner JB, Kosinski M, Ware JE. The feasibility of applying item response theory to measures of migraine impact: a re-analysis of three clinical studies. Qual Life Res. 2003;12:887–902
  10. Cella D, Yount S, Rothrock N, Gershon R, Cook K, Reeve B, et al. The Patient-Reported Outcomes Measurement Information System (PROMIS): progress of an NIH Roadmap cooperative group during its first two years. Med Care. 2007;45(5 Suppl. 1):S3–S11
  11. Hambleton RK. Emergence of item response modeling in instrument development and data analysis. Med Care. 2000;38(9 Suppl):II60–II65
  12. Institute of Medicine . Crossing the quality chasm: a new health system for the 21st century. Washington, DC: National Academy Press; 2001;
  13. Institute of Medicine . Rewarding provider performance: aligning incentives in medicare. Washington, DC: National Academies Press; 2006;
  14. Porter ME, Teisberg EO. Redefining health care. Creating value-based competition on results. Boston, MA: Harvard Business School Press; 2006;
  15. Lord FM, Novick MR. Statistical theories of mental test scores. Reading, MA: Addison-Wesley; 1968;
  16. Lord F. Some test theory for tailored testing. In:  Holtzman W editors. Computer-assisted instruction, testing, and guidance. New York, NY: Harper and Row; 1970;p. 139–183
  17. Ware JE, Kosinski M, Bjorner JB, Bayliss MS, Batenhorst A, Dahlof CG, et al. Applications of computerized adaptive testing (CAT) to the assessment of headache impact. Qual Life Res. 2003;12:935–952
  18. Jette AM, Haley SM, Tao W, Ni P, Moed R, Meyers D, et al. Prospective evaluation of the AM-PAC-CAT in outpatient rehabilitation settings. Phys Ther. 2007;87:385–398
  19. McHorney CA. Generic health measurement: past accomplishments and a measurement paradigm for the 21st century. Ann Intern Med. 1997;127(8 Pt 2):743–750
  20. Patrick DL, Chiang YP. Convening health outcomes methodologists. Med Care. 2000;38(9 Suppl):II3–II6
  21. Revicki DA, Cella DF. Health status assessment for the twenty-first century: item response theory, item banking and computer adaptive testing. Qual Life Res. 1997;6:595–600
  22. Dijkers MP. A computer adaptive testing simulation applied to the FIM instrument motor component. Arch Phys Med Rehabil. 2003;84:384–393
  23. Gardner W, Kelleher KJ, Pajer KA. Multidimensional adaptive testing for mental health problems in primary care. Med Care. 2002;40:812–823
  24. Haley SM, Ni P, Hambleton RK, Slavin MD, Jette AM. Computer adaptive testing improved accuracy and precision of scores over random item selection in a physical functioning item bank. J Clin Epidemiol. 2006;59:1174–1182
  25. Hart DL, Cook KF, Mioduski JE, Teal CR, Crane PK. Simulated computerized adaptive test for patients with shoulder impairments was efficient and produced valid measures of function. J Clin Epidemiol. 2006;59:290–298
  26. Hart DL, Mioduski JE, Stratford PW. Simulated computerized adaptive tests for measuring functional status were efficient with good discriminant validity in patients with hip, knee, or foot/ankle impairments. J Clin Epidemiol. 2005;58:629–638
  27. Hart DL, Mioduski JE, Werneke MW, Stratford PW. Simulated computerized adaptive test for patients with lumbar spine impairments was efficient and produced valid measures of function. J Clin Epidemiol. 2006;59:947–956
  28. Haley SM, Fragala-Pinkham M, Ni P. Sensitivity of a computer adaptive assessment for measuring functional mobility changes in children enrolled in a community fitness programme. Clin Rehabil. 2006;20:616–622
  29. Ware JE, Gandek B, Sinclair SJ, Bjorner J. Item response theory in computer adaptive testing: implications for outcomes measurement in rehabilitation. Rehabil Psychol. 2005;50:71–78
  30. Deutscher D, Hart DL, Dickstein R, Horn SD, Gutvirtz M. Implementing an integrated electronic outcomes and electronic health record process to create a foundation for clinical practice improvement. Phys Ther. 2008;88:270–285
  31. Hart DL, Connolly JB. Pay-for-performance for physical therapy and occupational therapy: medicare part B services. Grant #18-P-93066/9–01. Health & Human Services/Centers for Medicare & Medicaid Services. 2006;
  32. Ader DN. Developing the Patient-Reported Outcomes Measurement Information System (PROMIS). Med Care. 2007;45(5 Suppl. 1):S1–S2
  33. Haley SM, Coster WJ, Andres PL, Kosinski M, Ni P. Score comparability of short forms and computerized adaptive testing: simulation study with the activity measure for post-acute care. Arch Phys Med Rehabil. 2004;85:661–666
  34. Haley SM, Coster WJ, Andres PL, Ludlow LH, Ni P, Bond TL, et al. Activity outcome measurement for postacute care. Med Care. 2004;42(1 Suppl):I49–I61
  35. American Physical Therapy Association . Guide to physical therapist practice. Phys Ther. 2001;81:1–768
  36. Resnik L, Hart DL. Using clinical outcomes to identify expert physical therapists. Phys Ther. 2003;83:990–1002
  37. Werneke MW, Hart DL. Centralization phenomenon as a prognostic factor for chronic low back pain and disability. Spine. 2001;26:758–764
  38. Centers for Medicare and Medicaid Services . 2007 Physician Quality Reporting Initiative (PQRI). Physician quality measures. Centers for Medicare and Medicaid Services; 2007;
  39. Dobrzykowski EA, Nance T. The Focus On Therapeutic Outcomes (FOTO) Outpatient Orthopedic Rehabilitation Database: results of 1994-1996. J Rehabil Outcomes Meas. 1997;1:56–60
  40. Swinkels ICS, van den Ende CHM, de Bakker D, van der Wees J, Hart DL, Deutscher D, et al. Clinical databases in physical therapy. Physiother Theory Pract. 2007;23:153–167
  41. Binkley JM, Stratford PW, Lott SA, Riddle DL. The Lower Extremity Functional Scale (LEFS): scale development, measurement properties, and clinical application. North American Orthopaedic Rehabilitation Research Network. Phys Ther. 1999;79:371–383
  42. Alcock GK, Stratford PW. Validation of the Lower Extremity Functional Scale on athletic subjects with ankle sprains. Physiother Can. 2002;54:233–240
  43. Stratford PW. Getting more from the literature: Estimating the standard error of measurement from reliability studies. Physiother Can. 2004;56:27–30
  44. Stratford PW, Binkley JM, Watson J, Heath-Jones T. Validation of the LEFS on patients with total joint arthroplasty. Physiother Can. 2000;52:97–205
  45. Stratford PW, Hart DL, Binkley JM, Kennedy DM, Alcock GK, Hanna SE. Interpreting lower extremity functional status scores. Physiother Can. 2005;57:154–162
  46. World Health Organization . International Classification of Functioning, Disability and Health. Geneva: World Health Organization; 2001;
  47. Hart AC, Stegman MS. ICD-9-CM 2008 Expert. 6th edition. Salt Lake City, UT: Ingenix; 2007;
  48. Andrich D. A rating formulation for ordered response categories. Psychometrika. 1978;43:561–573
  49. Millsap RE, Everson HT. Methodology review: statistical approaches for assessing measurement bias. Appl Psychol Meas. 1993;17:287–334
  50. Crane PK, Hart DL, Gibbons LE, Cook KF. A 37-item shoulder functional status item pool had negligible differential item functioning. J Clin Epidemiol. 2006;59:478–484
  51. Thissen D, Mislevy RJ. Testing algorithms. In:  Wainer H editors. Computerized adaptive testing: a primer. 2nd edition. Mahwah, NJ: Lawrence Erlbaum Associates; 2000;p. 101–134
  52. Hart DL, Mioduski JE. CAT development and testing software user's guide. Knoxville, TN: FOTO, Inc; 2006;
  53. Lord FM. Applications of item response theory to practical testing problems. Hillsdale, NJ: Lawrence Erlbaum Associates; 1980;
  54. Linacre JM. Estimating measures with known polytomous item difficulties. Rasch Meas Trans. 1998;12:638
  55. Folk VG, Smith RL. Models for delivery of CBTs. In:  Mills CN,  Potenza MT,  Fremer JJ,  Ward WC editor. Computer-based testing building the foundation for future assessments. Mahwah, NJ: Lawrence Erlbaum Associates; 2002;p. 41–66
  56. Groll DL, To T, Bombardier C, Wright JG. The development of a comorbidity index with physical function as the outcome. J Clin Epidemiol. 2005;58:595–602
  57. Shapiro S, Wilk MB. An analysis of variance test for normality. Biometrika. 1965;52:591–611
  58. Jette DU, Jette AM. Physical therapy and health outcomes in patients with spinal impairments. Phys Ther. 1996;76:930–945
  59. Jette DU, Jette AM. Physical therapy and health outcomes in patients with knee impairments. Phys Ther. 1996;76:1178–1187
  60. Resnik L, Feng Z, Hart DL. State regulation and the delivery of physical therapy services. Health Serv Res. 2006;41(4 Pt 1):1296–1316
  61. Hart DL. The power of outcomes: FOTO industrial outcomes tool-initial assessment. Work. 2001;16:39–51
  62. Hart DL, Dobrzykowski EA. Influence of orthopaedic clinical specialist certification on clinical outcomes. J Orthop Sports Phys Ther. 2000;30:183–193
  63. Hart DL, Dobrzykowski EA. Impact of exercise history on health status outcomes in patients with musculoskeletal impairments. Orthop Phys Ther Clin North Am. 2000;9:1–16
  64. Beaton DE, Bombardier C, Katz JN, Wright JG. A taxonomy for responsiveness. J Clin Epidemiol. 2001;54:1204–1217
  65. Liang MH. Longitudinal construct validity: establishment of clinical meaning in patient evaluative instruments. Med Care. 2000;38(9 Suppl):II84–II90
  66. Wright JG, Young NL. A comparison of different indices of responsiveness. J Clin Epidemiol. 1997;50:239–246
  67. Kirshner B, Guyatt G. A methodological framework for assessing health indices. J Chronic Dis. 1985;38:27–36
  68. Riddle DL, Stratford PW, Binkley JM. Sensitivity to change of the Roland-Morris Back Pain Questionnaire: part 2. Phys Ther. 1998;78:1197–1207
  69. Beaton DE, Hogg-Johnson S, Bombardier C. Evaluating changes in health status: reliability and responsiveness of five generic health status measures in workers with musculoskeletal disorders. J Clin Epidemiol. 1997;50:79–93
  70. Liang MH. Evaluating measurement responsiveness. J Rheumatol. 1995;22:1191–1192
  71. Stratford PW, Binkley FM, Riddle DL. Health status measures: strategies and analytic methods for assessing change scores. Phys Ther. 1996;76:1109–1123
  72. Hsieh YW, Wang CH, Wu SC, Chen PC, Sheu CF, Hsieh CL. Establishing the minimal clinically important difference of the Barthel Index in stroke patients. Neurorehabil Neural Repair. 2007;21:233–238
  73. Stratford PW, Riddle DL. Assessing sensitivity to change: choosing the appropriate change coefficient. Health Qual Life Outcomes. 2005;3:23
  74. Crosby RD, Kolotkin RL, Williams GR. Defining clinically meaningful change in health-related quality of life. J Clin Epidemiol. 2003;56:395–407
  75. Jaeschke R, Singer J, Guyatt GH. Measurement of health status. Ascertaining the minimal clinically important difference. Control Clin Trials. 1989;10:407–415
  76. Lingard EA, Riddle DL. Impact of psychological distress on pain and function following knee arthroplasty. J Bone Joint Surg Am. 2007;89:1161–1169
  77. Tubach F, Ravaud P, Baron G, Falissard B, Logeart I, Bellamy N, et al. Evaluation of clinically relevant changes in patient reported outcomes in knee and hip osteoarthritis: the minimal clinically important improvement. Ann Rheum Dis. 2005;64:29–33
  78. Tubach F, Ravaud P, Beaton D, Boers M, Bombardier C, Felson DT, et al. Minimal clinically important improvement and patient acceptable symptom state for subjective outcome measures in rheumatic disorders. J Rheumatol. 2007;34:1188–1193
  79. Deyo RA, Centor RM. Assessing the responsiveness of functional scales to clinical change: an analogy to diagnostic test performance. J Chronic Dis. 1986;39:897–906
  80. Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology. 1982;143:29–36
  81. George SZ, Fritz JM, Bialosky JE, Donald DA. The effect of a fear-avoidance-based physical therapy intervention for patients with acute low back pain: results of a randomized clinical trial. Spine. 2003;28(23):2551–2560
  82. Chakravarty EF, Bjorner JB, Fries JF. Improving patient reported outcomes using item response theory and computerized adaptive testing. J Rheumatol. 2007;34:1426–1431
  83. Dodd BG, Koch WR, De Ayala RJ. Operational characheristics of adaptive testing procedures using the Graded Response Model. Appl Psychol Meas. 1989;13:129–143
  84. Martin M, Kosinski M, Bjorner JB, Ware JE, Maclean R, Li T. Item response theory methods can improve the measurement of physical function by combining the modified health assessment questionnaire and the SF-36 physical function scale. Qual Life Res. 2007;16:647–660
  85. Ware JE, Sherbourne CD. The MOS 36-item short-form health survey (SF-36). I. Conceptual framework and item selection. Med Care. 1992;30:473–483
  86. Hart DL, Wright BD. Development of an index of physical functional health status in rehabilitation. Arch Phys Med Rehabil. 2002;83:655–665
  87. Jette AM, Haley SM, Ni P. Comparison of functional status tools used in post-acute care. Health Care Financ Rev. 2003;24(3):13–24
  88. McHorney CA, Cohen AS. Equating health status measures with item response theory: illustrations with functional status items. Med Care. 2000;38(9 Suppl):II43–II59
  89. Steinberg L, Thissen D, Wainer H. Validity. In:  Wainer H editors. Computerized adaptive testing: a primer. 2nd edition. Mahwah, NJ: Lawrence Erlbaum Associates; 2000;p. 185–229
  90. Hart DL. Assessment of unidimensionality of physical functioning in patients receiving therapy in acute, orthopedic outpatient centers. J Outcome Meas. 2000;4:413–430
  91. Reeve BB, Hays RD, Bjorner JB, Cook KF, Crane PK, Teresi JA, et al. Psychometric evaluation and calibration of health-related quality of life item banks: plans for the Patient-Reported Outcomes Measurement Information System (PROMIS). Med Care. 2007;45(5 Suppl. 1):S22–S31
  92. American Medical Association . Current Procedural Terminology (CPT) 2007, professional edition. Chicago, IL: American Medical Association; 2006;

PII: S0895-4356(08)00029-2

doi: 10.1016/j.jclinepi.2008.01.005

Journal of Clinical Epidemiology
Volume 61, Issue 11 , Pages 1113-1124 , November 2008

Article Tools

* Image Usage & Resolution