Advantages of individual patient data analysis in systematic reviews
Article Outline
In the Meta-analyses and Systematic Review section of this issue, the question addressed is, “Why is individual patient data so rarely used in systematic reviews?” Individual patient data analysis is only used in 2% of published systematic reviews. As van Walraven points out in his commentary, individual patient data have many substantive and methodological advantages including outcome harmonization, analytic harmonization, and exploration of effectiveness variability. The Equity Group of the Campbell and Cochrane Collaborations [1] (www.equity.cochrane.org), for which one of the editors is co-convener, is now encouraging systematic reviewers to analyze the effectiveness differences within different types of disadvantaged groups (e.g., poor, uneducated, out of work, place of residence, social isolation, and gender); this really needs individual patient data to classify each person and their outcome by their disadvantaged status. One of the reasons for rarely using individual patient data is probably that the clinical epidemiology community has neither aggressively showcased good examples when teaching evidence-based medicine and critical appraisal to the clinician readers nor have we emphasized this when teaching in graduate programs and in theses; we should review this approach.
Abraham et al. challenge the traditional view that in surgery even well-designed nonrandomized comparative studies give flawed estimates of benefit and harms; they show that in trials of laparoscopic resection for colorectal cancer, the estimates of effect did not differ between nonrandomized comparative studies and randomized controlled trials. The comparison of trial designs needs to be assessed for other surgical interventions; the Journal of Clinical Epidemiology would welcome methodological manuscripts on this topic. In another surgically oriented paper, using the example of laparoscopic vs. small-incision cholecystectomy, Keus et al. make a case for using the systematic review to estimate the sample size needed in a new controlled trial to achieve tight enough confidence limits to be able to obtain conclusive evidence on which operative method is better. Sadatsafav et al., using the example of diagnosis of latent tuberculosis, demonstrate the benefits of combining traditional meta-analysis techniques with a latent-class model, allowing the analysis of data from heterogeneous sources. King et al. used meta-analysis to challenge the default interpretation of effect sizes as recommended by Cohen [2] of small (0.2), medium (0.5), and large (0.8) effect sizes when applied to a cancer-specific quality-of-life questionnaire (the Functional Assessment of Cancer Therapy-General [FACT-G]). In their analysis, mean differences yielded somewhat different small and medium effect sizes as compared with Cohen's generic guidelines. This has important sample size implications—with the lower estimates of “medium” effect sizes the sample size will need to be increased to achieve the usual alpha and beta criteria.
Schwarzer et al. compared two measures to estimate publication bias in meta-analyses that address the known problem of authors being more likely to submit trials with “significant” results for publication and journals being more likely to publish smaller trials if they have significant results. They compare the Copas selection model and the trim-and-fill method. They found the Copas method performed better empirically and recommend this to be estimated in all meta-analyses, but this is still controversial because although they differ in implementation, both are based on the concept of “making up trials” that are not known to actually exist. Horton et al. show some worrying results on the accuracy of data extraction for systematic reviews, even with expertise.
In the Original Articles section, a wealth of important methodological issues is also covered. Koschack et al. challenge the underlying conceptual framework of medication adherence and show that two commonly used questionnaires that measure medication adherence do not perform better than chance alone. They argue that the concept of “nonadherence” therefore needs to be rethought. Amiriana et al. present data on the concept of executive function, which they define as the capacity for behavioral self-regulation and show data to suggest that this may be important in predicting mortality. Mielenz et al. report on the psychometric properties, using item response theory (IRT), of different number of items on a scale measuring kinesophobia, defined as fear of movement, which is being used in a number of different conditions in which pain is a dominant symptom. van Nispena et al. also report on IRT and show that a multilevel IRT model can be applied to describe longitudinal dependent data. They claim that not only can group effects be estimated but also individual effects can be directly estimated from the multilevel IRT model. They suggest that instead of applying the usual longitudinal methods to compare pre- and posttreatment, multilevel IRT models offer an alternative. Finally, Hanmer et al. compare three methods tested to model SF-6D health utilities for health states involving comorbidity/co-occurring conditions in more than 90,000 seniors enrolled in Medicare managed care plans. The multiplicative and additive models both performed well in individuals with seven conditions or less. This attention to co- and multi-morbidity is a burgeoning and important area for the aging population that needs simple measures such as these. However, utilities and their use in making clinical decisions are still not fully accepted. This needs thinking through by the different disciplines involved if utility-based instruments are to be worth a lot of investment in more research resources. We would welcome correspondence on this topic from readers of the Journal of Clinical Epidemiology.
References
PII: S0895-4356(09)00390-4
doi:10.1016/j.jclinepi.2009.12.005
© 2010 Published by Elsevier Inc.
