If you don't remember your password, you can reset it by entering your email address and clicking the Reset Password button. You will then receive an email that contains a secure link for resetting your password
If the address matches a valid account an email will be sent to __email__ with instructions for resetting your password
Swiss Paraplegic Research, Guido A. Zäch Strasse 4, 6207 Nottwil, SwitzerlandDepartment of Health Sciences and Medicine, University of Lucerne, Frohburgstrasse 3, 6002 Lucerne, Switzerland
Swiss Paraplegic Research, Guido A. Zäch Strasse 4, 6207 Nottwil, SwitzerlandCenter for Rehabilitation in Global Health Systems, Department of Health Sciences and Medicine, University of Lucerne, Frohburgstrasse 3, 6002 Lucerne, Switzerland
Swiss Paraplegic Research, Guido A. Zäch Strasse 4, 6207 Nottwil, SwitzerlandDepartment of Health Sciences and Medicine, University of Lucerne, Frohburgstrasse 3, 6002 Lucerne, SwitzerlandFaculty of Applied Health and Social Sciences, Technical University of Applied Sciences Rosenheim, Hochschulstraße 1, 83024 Rosenheim, Germany
The study aimed to explore existing prediction models of functioning in spinal cord injury (SCI).
Study Design and Setting
The databases PubMed, EBSCOhost CINAHL Complete, and IEEE Xplore were searched for relevant literature. The search strategy included published search filters for prediction model and impact studies, index terms and keywords for SCI, and relevant outcome measures able to assess functioning as reflected in the International Classification of Functioning, Disability and Health (ICF). The search was completed in October 2020.
Results
We identified seven prediction model studies reporting twelve prediction models of functioning. The identified prediction models were mainly envisioned to be used for rehabilitation planning, however, also other possible applications were stated. The method predominantly used was regression analysis and the investigated predictors covered mainly the ICF components of body functions and activities and participation, next to characteristics of the health condition and health interventions.
Conclusion
Findings suggest that the development of prediction models of functioning for use in clinical practice remains to be fully exploited. By providing a comprehensive overview of what has been done, this review informs future research on prediction models of functioning in SCI and contributes to an efficient use of research evidence.
Spinal cord injury (SCI) is a chronic health condition devastatingly affecting a person's life in a variety of ways. The structural damage to the spinal cord and the resulting loss of neurologic functions adversely affects the ability of a person to perform simple and complex activities and to participate in community and major life areas [
]. After the injury, persons with SCI go through an extensive rehabilitation process to live independently with the health condition: from intensive care and inpatient rehabilitation to outpatient specialized care after returning to the community. The World Health Organization (WHO) refers to the lived experience of a health condition as `functioning' [
]. The concept of functioning, as described in WHO's International Classification of Functioning, Disability and Health (ICF), includes different components – body functions and body structures as well as activities and participation – which interact with each other and are outcomes of the interaction between a health condition and environmental and personal contextual factors. Against this background, the objective of rehabilitation after SCI can be formulated as the optimization and maintenance of a person's functioning [
]. In order to achieve this objective, comprehensive and relevant functioning information is essential to guide rehabilitation planning and management, individual clinical care and decision making.
Prediction research aims to enhance individual health and health care practice by investigating and improving the diagnosis or prognosis of a specific health condition [
] (1) predictor finding studies, (2) prediction model studies, and (3) impact studies. Predictor finding studies generally aim to explore or identify which variables within a set of candidate predictors are independently associated with a specific outcome. Prediction model studies aim to develop and/or externally validate (with or without updating) a multivariable prediction model for use in medical or clinical practice. Impact studies build on a developed and validated prediction model and aim to assess the impact of the use of such a model in a specific context or setting compared to not using it. Prediction model development, validation and impact studies correspond with the phases, which prediction models for use in practice usually have to undergo in their development process [
]. The development of prediction models has gained increasing attention by the recognition of evidence-based health care and the uptake of new statistical methods in the health sciences and clinical epidemiology.
In rehabilitation research, the role of functioning as key health indicator complementing mortality and morbidity [
] poses the question of how prediction research, and specifically prediction models, can improve the use of functioning information for practice. In SCI literature, various efforts have been undertaken to develop and/or validate prediction models for outcomes related to specific aspects of functioning, such as ambulation, [
A simplified clinical prediction rule for prognosticating independent walking after spinal cord injury: a prospective study from a Canadian multicenter spinal cord injury registry.
Highlighting discrepancies in walking prediction accuracy for patients with traumatic spinal cord injury: an evaluation of validated prediction models using a Canadian Multicenter Spinal Cord Injury Registry.
Prediction of bladder outcomes after ischemic spinal cord injury: A longitudinal cohort study from the European multicenter study about spinal cord injury.
]. What remains to be investigated is how functioning, as a multidimensional concept is reflected in current prediction models across the corresponding development phases depicted by development, validation and impact studies in the field of SCI rehabilitation. Therefore, the objective of this scoping review is to explore existing prediction models of functioning in SCI. Specifically, the review aims to (1) identify prediction models of functioning in SCI, (2) examine their content by using the ICF as a reference language, (3) examine their use from a systems perspective, and (4) document which methods were used to develop them. The scoping review will shed light on current research gaps as well as on promising directions for future developments and improvements of prediction models of functioning for SCI.
What is new?
Key findings
•
Identification of seven prediction model studies reporting twelve prediction models of functioning in SCI; no impact study was identified.
What this adds to what is known?
•
The development of prediction models of functioning in SCI is still in its infancy. This review highlights potential future directions in the development of prediction models in the field of SCI rehabilitation with regards to content, use and methods.
What is the implication, what should change now?
•
Functioning, as outcome of the identified models, was measured with the FIMTM or the SCIM. The investigated predictors covered mainly body functions, activities and participation, characteristics of the health condition or health interventions. The integration of a broad range of potential predictors including imaging, biomarkers, and genetics, as well as predictors covering body structures and contextual factors remains to be investigated.
•
The method predominantly used was linear regression analysis. The application and usefulness of other methods such as machine learning techniques need to be further investigated and its potential merit compared to current methods.
•
The identified prediction models were intended to be used for guidance in rehabilitation planning, patient counselling, financial aspects related to the reduction of costs by guided management strategies, and improvements in clinical trial designs. To delineate the value of prediction models for the field of SCI rehabilitation in detail, further research is needed related to validation and impact assessment of prediction models.
2. Methods
The scoping review followed the methodological framework of Arksey and O'Malley [
]. An unpublished review protocol was developed and agreed upon by all authors prior to conducting the review and is available from the authors on request. The reporting followed the Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews [
]. The databases were chosen to cover literature from a broad spectrum of rehabilitation research topics including clinical and biomedical sciences, nursing and allied health, as well as biomechanical and engineering sciences. We did not explicitly search for grey literature.
The search strategy was defined in an iterative fashion [
] which is available on PubMed via the search filters for "Clinical Queries", 2) an update to the strategy in step one in the form of the Teljeur/Murphy Inclusion Filter introduced by Keogh et al. [
] and adapted by the authors of this study, 3) index terms and keywords for SCI, and 4) relevant outcome measures able to assess the lived experience of health in persons with SCI as operationalized by functioning. The latter were identified by the development of an initial list based on literature [
German-speaking Medical SCI Society (DMGP) Ergebniserhebung Guideline Development Group A systematic review of outcome measures in initial rehabilitation of individuals with newly acquired spinal cord injury: providing evidence for clinical practice guidelines.
Using the refined ICF Linking Rules to compare the content of existing instruments and assessments: a systematic review and exemplary analysis of instruments measuring participation.
] and feedback by scholars in the field about the most important measures to consider, given the scope of this study. Included languages were German and English, no limits were chosen with regards to the publication date. The search strategy was developed using PubMed and afterwards translated and adapted to the particularities of the identified other databases. The full search strategy for all databases can be found in the Supplemental Table 2. The search was completed on October 12th 2020.
2.2 Study selection
Eligibility was formulated according to in- and exclusion criteria for title/abstract and full-text screening separately (see Table 1). Underlying the eligibility criteria are the different types of prediction research explained in the introduction. Prediction models are thereby understood as ``tools that combine multiple predictors by assigning relative weights to each predictor to obtain a risk or probability'' [
]. Other notions include (clinical) prediction rules, probability assessments, decision rules or risk scores. In accordance with the objective of this review, only models were included that predicted functioning: Outcome variables included in the studies had to reflect different domains of functioning (classified as chapters in the ICF), but at least two chapters of activities and participation. Published conference proceedings in the biomechanical and engineering sciences were considered as original publications.
Table 1Eligibility criteria according to title/abstract screening and full-text screening
Inclusion and exclusion criteria for title/abstract screening
Inclusion criteria:
•
Primary study
•
Prediction model study or impact study
•
Study includes at least one variable (predictor and/or outcome) assessed with a measure of the lived experience of health as operationalised by functioning, which reflects two or more chapters of activities and participation as described in the ICF
•
Study population includes males and/or females with SCI (traumatic and/or non-traumatic)
•
Publication language is English or German
Exclusion criteria:
•
Animal study
•
Paediatric study
•
Predictor finding study
•
Prediction model study or impact study with mixed-diagnosis populations
•
Study population includes SCI as a complication
•
Study includes mortality as solely outcome
Inclusion and exclusion criteria for full-text screening
Inclusion criteria:
•
Study includes measure of functioning as outcome variable
Exclusion criteria:
•
Study includes measure of functioning as predictor variable only
•
Study includes as outcome variable only single items or subscales of a measure of functioning, which no longer reflect two or more chapters of activities and participation as described in the ICF
•
Study with outcome assessed/evaluated within the acute rehabilitation setting
Abbreviations. ICF, International Classification of Functioning, Disability and Health; SCI, spinal cord injury.
] for screening of titles and abstracts. A random sample incorporating 50 articles of the records were screened independently by two reviewers (JH, BP) in light of the eligibility criteria to determine whether an article is relevant. If the agreement in decisions for article in- or exclusion of the reviewers was acceptable (>90%), one reviewer continued to screen the remaining articles (JH). Otherwise, a new random sample of the same size was screened independently by the two reviewers. Disagreement was solved by discussions and the procedure was repeated until an acceptable agreement was reached.
Before starting the full-text screening, the eligibility criteria were revisited and further detailed by the study team. Subsequently, full-texts were screened by one reviewer (JH) and in the case of ambiguity, discussed with a second reviewer (BP). After full-text screening, an additional hand search was conducted. The database findings, screening and references were organized with EndNote [
] were entered into a Microsoft Excel sheet and complemented by elements of the checklists for Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies (CHARMS) [
] was applied. This method allows to link the content of outcomes or predictors included in the respective prediction models to the ICF as a reference model, and thus enables the comparison of outcomes and predictors contained in different prediction models. The linking process entails the linking at the conceptual and the classification level. For the purpose of this review, outcomes and predictors reported in the identified studies were extracted and linked if possible at chapter-level of the ICF. The ICF Research Branch (https://www.icf-research-branch.org) was contacted to request existing linking results of specific outcomes and predictors. To examine the envisioned use and implications of the identified prediction models, micro (patient-provider interaction), meso (service provision and payment) and macro (policies and programs) system levels were used as framework of reference. To document the methods used to develop the identified prediction models, the respective author's description used within the article were extracted together with the stated argumentation for its use, as well as stated advantages and disadvantages.
The data extraction was performed by one reviewer (JH) and cross-checked by a second reviewer (BP). The results of the scoping review were arranged in tabular format and discussed narratively.
3. Results
3.1 Study identification
In total, 2378 articles were retrieved through database searching and after screening the titles and abstracts of 1851 articles and the full-texts of 234 articles, seven eligible studies were identified for inclusion in the scoping review [
Relationship between clinical assessments of function and measurements from an upper-limb robotic rehabilitation device in cervical spinal cord injury.
]. The corresponding flow diagram of the screening process is presented in Figure 1.
Figure 1Flow diagram of the scoping review. Note that the reasons for full-text exclusion are not mutually exclusive. Figure adapted from Moher et al. 2009
For the title and abstract screening, in total three random samples were screened independently by the two reviewers until acceptable agreement was reached. The specific agreement levels reached for each sample were 78%, 86%, and 94%, respectively. Main reason for disagreement was the challenging distinction between predictor finding studies and prediction model studies. Following the framework of Kent et al. [
] the distinction should be based on the study aim. However, often authors did not clearly state the study aim, which was also reported by authors who conducted reviews on prediction models previously [
]. If a study aim was not clearly stated or unsure, studies were nevertheless included for full-text screening if they described a functioning outcome, or mentioned some form of model performance or accuracy assessment.
As the eligibility criteria for the full-text screening were revisited, for prediction model development studies the criteria, that studies need to include an internal validation of the prediction models to be eligible for this review, was decided. This decision was based on the recommendation of the TRIPOD statement for prediction model development studies to include some form of internal validation. In addition, this decision enhanced the consistency in the distinction between prediction model and predictor finding studies.
In the hand search we applied the following criteria: 1) publications based on identified SCI cohorts, trials or research projects (European Multicenter Study about Spinal Cord Injury, Rick Hansen Spinal Cord Injury Registry, Spinal Cord Injury Model System, SCIRehab) were specifically searched for in PubMed, and 2) the identified eligible studies were checked for updates using the 'Cited-by'-function of PubMed.
3.3 Characteristics of the included studies
The basic characteristics of the seven included prediction model studies are shown in Table 2. Six studies [
Relationship between clinical assessments of function and measurements from an upper-limb robotic rehabilitation device in cervical spinal cord injury.
Relationship between clinical assessments of function and measurements from an upper-limb robotic rehabilitation device in cervical spinal cord injury.
]. The mean age of the study populations under investigation ranged from 43 (SD=18) to 60 (SD=16) years and the population samples focused on traumatic aetiology and tend to include predominantly men and persons with tetraplegia. No impact studies were found.
Table 2Overview of included prediction model studies.
Abbreviations. AIS, American Spinal Injury Association Impairment Scale; LOCF, last observation carried forwards; NA, not available; NOCB, next observation carried backwards; SD, standard deviation; UK, United Kingdom; USA, United States of America.
Note. Estimates and percentages have been rounded to zero decimal places for the purpose of this review.
a If AIS grade was reported at several time points, the earliest was chosen for this overview;
In total, the seven included articles described 12 prediction models of functioning. Table 3 shows the identified models, their specific outcomes, investigated predictors and the corresponding linking to the ICF. The functioning outcome variables used in the prediction models all related to the two instruments Spinal Cord Independence Measure (SCIM) and Functional Independence Measure (FIMTM), which both are assessing functional independence of a person in daily life, specifically focusing on self-care, mobility, and bladder and bowel management. The time scope for prediction ranged up to one year after injury. Predictors were assessed during early acute phase and up to one month after injury. Investigated predictor variables described concepts covered by the ICF components body functions, and activities and participation. Predictors that could not be linked to the ICF mainly described characteristics of the health condition or health interventions. With regards to their intended or envisioned use, all prediction models were assigned to the micro system level (e.g. guidance in rehabilitation planning, goal setting and patient care) [
Relationship between clinical assessments of function and measurements from an upper-limb robotic rehabilitation device in cervical spinal cord injury.
Relationship between clinical assessments of function and measurements from an upper-limb robotic rehabilitation device in cervical spinal cord injury.
]. The reported statistical methods for the development of the prediction models were mostly regression analyses (linear and logistic), one study reported the use of machine learning methods, specifically regression tree analysis [
The respective models differ according to the used regression method and predictor selection (linear regression and significance criteria used for models 3 and 4 vs. generalized linear regression and elastic net penalization used for models 1 and 2).
The two models differ according to the used coding scheme of FIMTM and corresponding regression method (discrete score and linear regression model vs. dichotomization according to the achievement of a score of at least 6 in all FIMTM motor score items and logistic regression model);
Only prediction models of functioning outcomes are reported for the purpose of this review;
SCIM III, total score
Inpatient rehabilitation
X
-
-
-
X
X
Hand range of motion, x direction
All predictor variables were assessed within two weeks of the SCIM III assessment (before or after)
X
X
Hand range of motion, y direction
X
Hand range of motion, z direction
X
X
Joint range of motion, angle 1
X
Joint range of motion, angle 2
X
Joint range of motion, angle 3
X
Joint range of motion, angle 4
X
Joint range of motion, angle 5
X
Movement mean jerk over task duration
X
Movement mean velocity over task duration
X
Number of changes in hand's trajectory direction, normalized by task length
X
Range of grip pressure
X
X
Ratio of mean to maximum velocity over task duration
X
Skewness of grip pressure
X
X
Abbreviations. ASIA, American Spinal Injury Association examination according to the International Standards for Neurological Classification of Spinal Cord Injury; b, body functions; d, activities and participation; e, environmental factors; FIMTM, Functional Independence Measure; ICF, International Classification of Functioning, Disability and Health; ISS, Injury Severity Score; MRI, magnetic resonance imaging; nc, not covered in the ICF; nc_hc, not covered in the ICF, health condition; nc_ICHI, not covered in the ICF, health intervention (International Classification of Health Interventions); NA, not available; nd, not defined; pf, personal factors (not classified in the ICF); s, body structures; SCIM III, Spinal Cord Independence Measure version three; TBI, traumatic brain injury; WISCI II, Walking Index for Spinal Cord Injury version two.
a In total, 20 variables were tested, of which 3 entered the final model;
b In total, 112 variables were tested, of which none entered the final model;
c In total, 19 variables were tested, of which 1 entered the final model;
d Only prediction models of functioning outcomes are reported for the purpose of this review;
e Mean time of blood test was 31 days (SD = 30 days) post-injury;
f Mean assessment time of SCIM III was 69.8 days (SD = 55.6 days) from admission, and mean time between injury and admission was 45.2 days (SD = 60.8);
g Mean assessment time of SCIM III was 123.4 days (SD = 58.2 days) from admission, and mean time between injury and admission was 45.2 days (SD = 60.8);
h The two models differ according to the used coding scheme of FIMTM and corresponding regression method (discrete score and linear regression model vs. dichotomization according to the achievement of a score of at least 6 in all FIMTM motor score items and logistic regression model);
i The respective models differ according to the used regression method and predictor selection (linear regression and significance criteria used for models 3 and 4 vs. generalized linear regression and elastic net penalization used for models 1 and 2).
We identified seven prediction model studies reporting twelve prediction models of functioning. No corresponding impact studies were found. This suggests that the development of prediction models of functioning and their use in practice is not fully exploited. In order to improve prediction models in SCI, it might be helpful to contrast current models with recent suggestions and examples from other health conditions.
All functioning outcome variables used in the identified prediction models related either to SCIM or FIMTM. Predictor variables covered the ICF components body functions (e.g. assessed by the American Spinal Injury Association examination), and activities and participation (e.g. assessed by SCIM). Other predictors described characteristics of the health condition (e.g. level of injury, complications) or of health interventions (e.g. delay to surgery). Only few studies investigated predictors such as blood measures, [
Relationship between clinical assessments of function and measurements from an upper-limb robotic rehabilitation device in cervical spinal cord injury.
Gap Analysis Regarding Prognostication in Neurocritical Care: A Joint Statement from the German Neurocritical Care Society and the Neurocritical Care Society.
] which both identified gaps in the investigation of a broad range of possible predictors including biological and physical, as well as psychosocial measures, and especially in the use of directly observable predictors such as imaging, biomarkers, and genetics. In terms of covered ICF components, the integration of body structures and contextual factors in prediction models remains scarce. Despite the use of the ICF as a frame of reference in the study and the consistency of using FIMTM and SCIM as outcomes, the comparability of the findings with regards to selected predictors is limited due to the application of different variable coding schemes such as dichotomized, discrete, or interval scores. Moreover, the comparability of the identified prediction models is further hampered by the heterogeneity of the study populations and settings, as well as by the different time points of predictor and outcome measurements. Further information standards are needed to enhance the interoperability of functioning outcomes or existing standards, such as the ICF or the SCI Data Set actually used in research and practice.
The method most often used in these identified prediction models was linear regression analysis. Only two identified studies were multi-centre studies and the respective population samples focused on traumatic aetiology and tend to include predominantly men and persons with tetraplegia, which limits the generalizability of the developed prediction models. Due to the complex and multidimensional nature of functioning in SCI, prediction models based on new methods such as machine learning techniques are promising and may allow a dynamic and real time modelling of interactions among a variety of predictors [
]. To do so, large data sets, ideally designed specifically for prediction research, including a broad variety of predictors and appropriately reflecting the population under study are needed [
The identified prediction models were intended for clinical purposes including guidance in individual rehabilitation planning, financial aspects related to the reduction of costs by guided management strategies, patient counselling, as well as for research purposes including the improvement of clinical trial designs, which are in line with other prediction research studies in SCI [
]. To delineate the value of prediction models for the field of SCI rehabilitation in detail, validation and impact assessment of prediction models require further research.
4.1 Limitations
There are some limitations to this review. Firstly, scoping reviews aim to give an overview of existing evidence on a given topic, regardless of the quality of the reviewed literature [
]. Since we did not assess the quality of the included studies, we are not able to make any statement about the performances, the usefulness or applicability of the presented prediction models for practice. Secondly, the search strategy specifically included common instruments assessing functioning and used in SCI. We do no claim this list to be complete and it might be the case that prediction model studies were missed because their instruments were not included in our search strategy. Thirdly, although our search strategy based on published search filters for prediction model and impact studies, these filters have been shown to low perform for the search of impact studies [
]. Furthermore, despite the absence of relevant impact studies, prediction models of functioning might be developed and implemented locally and not published internationally. Fourthly, the eligibility criteria understand functioning outcomes as variables covering at least two chapters of the ICF component activities and participation. Fifthly, the present review only includes prediction model studies which performed at least some kind of internal validation. Although internal validation is strongly recommended in prediction model development, this eligibility criterion lead to the exclusion of studies [
] about prediction model development which did not intend or failed for some reason to perform an internal validation. Such studies might also include valuable details to inform the development of prediction models in the future. For example, they might include information on potentially important predictor variables to consider in the development of prediction models of functioning, such as different neurophysiological variables as investigated by Hupp et al. [
]. Sixthly, we considered conference proceedings from the engineering sciences as original publications. However, these proceedings were often shorter than ordinary journal articles and thus, provided less information for the full-text screening and the categorization of excluded articles. Lastly, the authors had primarily expertise in the field of health sciences and less so in biomechanical and engineering sciences.
5. Conclusion
This scoping review sheds light on existing prediction models of functioning in SCI and highlights their content, use cases, and development methods. Findings suggest that the development of prediction models of functioning for use in clinical practice remains to be fully exploited. However, we believe that SCI with its many different functioning aspects concerned and its life-long perspective and requirement for health and social services across the entire continuum of care is an excellent learning example for the development of prediction models of functioning. By providing a comprehensive overview of what has been done, we hope to inform future research on prediction models of functioning in SCI, including the development of new prediction models for specific purposes or the external validation and improvement of existing ones, and contribute to an efficient and meaningful synthesis and use of research evidence.
Acknowledgements
The authors would like to thank Jerome Bickenbach for his valuable feedback on the manuscript, and Hildegard Oswald for her support in the search strategy development and its translation between the different databases.
This review is part of the cumulative dissertation of Jsabel Hodel which was conducted within the Swiss National Science Foundation's National Research Programme ``Smarter Health Care'' (NRP74) and the project ``Enhancing continuous quality improvement and supported clinical decision making by standardized reporting of functioning''.
A simplified clinical prediction rule for prognosticating independent walking after spinal cord injury: a prospective study from a Canadian multicenter spinal cord injury registry.
Highlighting discrepancies in walking prediction accuracy for patients with traumatic spinal cord injury: an evaluation of validated prediction models using a Canadian Multicenter Spinal Cord Injury Registry.
Prediction of bladder outcomes after ischemic spinal cord injury: A longitudinal cohort study from the European multicenter study about spinal cord injury.
German-speaking Medical SCI Society (DMGP) Ergebniserhebung Guideline Development Group
A systematic review of outcome measures in initial rehabilitation of individuals with newly acquired spinal cord injury: providing evidence for clinical practice guidelines.
Using the refined ICF Linking Rules to compare the content of existing instruments and assessments: a systematic review and exemplary analysis of instruments measuring participation.
Relationship between clinical assessments of function and measurements from an upper-limb robotic rehabilitation device in cervical spinal cord injury.
Gap Analysis Regarding Prognostication in Neurocritical Care: A Joint Statement from the German Neurocritical Care Society and the Neurocritical Care Society.
Competing interest: The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Presentation of this material: None.
Funding: The study was funded by the Swiss Paraplegic Research.
00229-8&id=gr1.jpg){kind=link}