On the validity of meta-analyses: exhaustivity must be warranted, exclusion of duplicate patients too
Article Outline
Exhaustivity of the selection of primary studies is a challenge in systematic reviews [1]. In contrast, including duplicate observations should also be a concern, a difficulty rarely addressed in papers. We would like to share here evidence that this may be an important issue and that excluding duplicate publications might not be enough.
Our team carried out in 2007, a meta-analysis of the diagnostic accuracy of transient elastography [2], a noninvasive imaging technique to stage liver fibrosis by measuring liver stiffness [3]. To overcome limitations because of aggregated data, we undertook the Transient Elastography Individual Patient Data Meta-analysis Study, which is currently in progress.
We contacted the authors of the 17 eligible full-length articles: 13 sent us their database, for a total of 2,788 patients. Identification of possible duplicates was based on dates of birth, liver biopsy, and transient elastography. The four French databases that belong to the manufacturer (Echosens, Paris, France) [4], [5], [6], [7] included 733 observations, of which 49 were duplicates. The four databases from the other French studies [8], [9], [10], [11] included 1,511 observations, of which 335 were duplicates. After merging the two resulting databases, the final database included 1,860 observations (684
+1,176) among which 520 additional duplicates were detected. No duplicate observations were found in other databases. After further exclusion of 55 observations coming from same patients, our meta-analysis will therefore deal with 1,829 different independent observations.
In a recent meta-analysis of transient elastography performances [12], based on aggregated data, the authors have sought to achieve exhaustivity by selecting both complete studies and congress abstracts. They considered that full-length articles did not include duplicates, and retained data from a congress abstract only if they had not been published elsewhere. No information was reported on results of possible contacts with authors. Our results indicate that data from eight studies that contributed to at least 2,244 observations of Hepatitis C virus (HCV) patients in this meta-analysis included at least 904 duplicates (40%).
In addition to already documented limitations, diagnostic accuracy meta-analyses conducted on aggregated data may include duplicates that are likely to bias pooled estimated accuracy if they are numerous. If duplicate publications might be detected by different means [13], duplicate observations published through different studies may be missed. Our recommendations are (1) to precisely record periods of inclusion when several studies are carried out in the same centers (it is indeed recommended, for additional reasons, by the Standards for the Reporting of Diagnostic Accuracy Studies statement) [14], to check possible period overlaps and the eventuality of same observations being included in several studies, (2) to systematically contact the authors of full-length articles, but also of abstracts when they are incorporated, in an attempt to identify possible duplicates, and (3) to promote registration of diagnostic clinical studies, as it was done in the field of therapeutic clinical trials [15]. Finally, diagnostic accuracy meta-analysis on individual patient data should be further developed and increasingly used to overcome both this problem and other usual limits of aggregated data meta-analysis [16].
References
- . How important are comprehensive literature searches and the assessment of trial quality in systematic reviews? Empirical study. Health Technol Assess. 2003;7:1–76
- . Méta-analyse des performances diagnostiques de l'élastométrie impulsionnelle (Fibroscan) dans l'évaluation de la fibrose hépatique chez les patients atteints d'une hépatite chronique C, [s2o01] Rev Epidemiol Sante Publique. 2008;56:S70
- . Non-invasive evaluation of liver fibrosis using transient elastography. J Hepatol. 2008;48:835–847
- Diagnosis of hepatic fibrosis and cirrhosis by transient elastography in HIV/hepatitis C virus-coinfected patients. J Acquir Immune Defic Syndr. 2006;41:175–179
- Accuracy of liver stiffness measurement for the diagnosis of cirrhosis in patients with chronic liver diseases. Hepatology. 2006;44:1511–1517
- Transient elastography: a new noninvasive method for assessment of hepatic fibrosis. Ultrasound Med Biol. 2003;29:1705–1713
- Noninvasive assessment of liver fibrosis by measurement of stiffness in patients with chronic hepatitis C. Hepatology. 2005;41:48–54
- Prospective comparison of transient elastography, Fibrotest, APRI, and liver biopsy for the assessment of fibrosis in chronic hepatitis C. Gastroenterology. 2005;128:343–350
- Diagnosis of cirrhosis by transient elastography (FibroScan): a prospective study. Gut. 2006;55:403–408
- Features associated with success rate and performance of fibroscan measurements for the diagnosis of cirrhosis in HCV patients: a prospective study of 935 patients. J Hepatol. 2007;46:628–634
- Liver stiffness measurement versus clinicians' prediction or both for the assessment of liver fibrosis in patients with chronic hepatitis C. Am J Gastroenterol. 2006;101:2744–2751
- Performance of transient elastography for the staging of liver fibrosis: a meta-analysis. Gastroenterology. 2008;134:960–974e8
- . Different patterns of duplicate publication: an analysis of articles used in systematic reviews. JAMA. 2004;291:974–980
- Towards complete and accurate reporting of studies of diagnostic accuracy: the STARD initiative. Ann Intern Med. 2003;138:40–44
- Is this clinical trial fully registered?—A statement from the International Committee of Medical Journal Editors. N Engl J Med. 2005;352:2436–2438
- . Systematic reviews with individual patient data meta-analysis to evaluate diagnostic tests. Eur J Obstet Gynecol Reprod Biol. 2003;108:121–125
Conflicts of interest: The Sterring Committee of Transient Elastography Individual Patient Data Meta-analysis Study includes authors of publications reporting primary studies. One of the author's database (Victor de Lédinghen) is the property of Fibroscan® manufacturer (Echosens, France).
PII: S0895-4356(09)00302-3
doi:10.1016/j.jclinepi.2009.07.021
© 2010 Elsevier Inc. All rights reserved.
