Journal of Clinical Epidemiology

Editorial Board

2012-03-01

Why are reporting guidelines not more widely used by journals?

Peter Tugwell, André Knottnerus, Leanne Idzerda — 2012-03-01

Reporting guidelines have become almost an industry in itself. Vandenbroucke raised some important issues when the JCE co-published the STREGA guidelines in 2009. He asked what exactly should the role of publication guidelines be and who needs them. An article in this issue by Larson and Cortazal contributes to this debate by providing an update on the development and adoption of general publication guidelines for various study designs. They provide examples of guidelines adapted for specific topics and recommend next steps. To assess the extent to which guidelines are being used and cited, they searched PubMed for the years after the first publication of each guideline through December 2010. A useful summary table of guidelines shows the number of times that published guidelines have been cited; this ranges from 2 to 565 citations. The authors recommend more aggressive promotion of guideline adoption among journals, educating peer reviewers in their use, and incorporating guideline use into the curriculum of medical, nursing, and public health schools.

Quantifying the unquantifiable

Gerald Gartlehner — 2011-12-12

Applicability has been described as “inferences about the extent to which a causal relationship holds over variations in persons, settings, treatments, and outcomes” . Consequently, to determine the applicability of any research result to a population of interest in a given setting, one has to take two aspects into consideration: first, whether, and to what extent, differences in patient characteristics, settings, treatments, and outcomes are present between a given body of evidence and a population of interest and second, whether existing differences can act as effect modifiers and have the potential to alter or even reverse the magnitude of an observed treatment effect.

Classification systems to improve assessment of risk of bias

Isabelle Boutron, Philippe Ravaud — 2012-03-01

Systematic reviews and meta-analyses are a cornerstone of comparative effectiveness research . Clinical decision makers often rely on the results of systematic reviews to develop guidelines. The strength of these methods relies on a scientifically rigorous approach that identifies, selects, and appraises all studies performed in a specific research area.

Publication guidelines need widespread adoption

Elaine L. Larson, Manuel Cortazal — 2011-10-17

Abstract: Objective: During the past two decades teams of researchers and editors have developed a variety of publishing guidelines to improve the quality of published research reports. Journals and editorial groups have adopted many of these guidelines. Whereas some guidelines are widely used, others have yet to be generally applied, thwarting attainment of consistent reporting among published research reports. The aim of this study is to describe the development and adoption of general publication guidelines for various study designs, provide examples of guidelines adapted for specific topics, and recommend next steps.Study Design and Setting: We reviewed generic guidelines for reporting research results and surveyed their use in PubMed and Science Citation Index.Results: Existing guidelines cover a broad spectrum of research designs, but there are still gaps in topics and use. Appropriate next steps include increasing use of available guidelines and their adoption among journals, educating peer reviewers on their use, and incorporating guideline use into the curriculum of medical, nursing, and public health schools.Conclusion: Wider adoption of existing guidelines should result in research that is increasingly reported in a standardized, consistent manner.

Medical journal editors lacked familiarity with scientific publication issues despite training and regular exposure

Victoria S.S. Wong, Michael L. Callaham — 2011-11-10

Abstract: Objective: To characterize medical editors by determining their demographics, training, potential sources of conflict of interest (COI), and familiarity with ethical standards.Study Design and Setting: We selected editors of clinical medical journals with the highest annual citation rates. One hundred eighty-three editors were electronically surveyed (response rate, 52%) on demographics and experiences with editorial training, publication ethics, industry, and scientific publication organizations.Results: Editors reported formal (76%) and informal (89%) training in medical editing topics. Most editors saw publication ethics issues (e.g., authorship, COIs) at least once a year. When presented with four questions about editorial issues discussed in commonly cited authoritative policy sources, performance was poor on topics of authorship (30% answered correctly), COI (15%), peer review (16%), and plagiarism (17%). Despite this, confidence level in editorial skills on a Likert scale from the beginning to the end of the survey dropped only slightly from 4.2 to 3.9 (P<0.0001).Conclusion: Our study presents a current look at editors of major clinical medical journals. Most editors reported training in medical editing topics, saw ethical issues regularly, and were aware of scientific publication organizations, but their knowledge of four common and well-disseminated publication ethics topics appears poor.

A critical review of methods used to determine the smallest worthwhile effect of interventions for low back pain

2011-10-20

Abstract: Objective: To critically and systematically review methods used to estimate the smallest worthwhile effect of interventions for nonspecific low back pain.Study Design and Setting: A computerized search was conducted of MEDLINE, CINAHL, LILACS, and EMBASE up to May 2011. Studies were included if they were primary reports intended to measure the smallest worthwhile effect of a health intervention (although they did not need to use this terminology) for nonspecific low back pain.Results: The search located 31 studies, which provided a total of 129 estimates of the smallest worthwhile effect. The estimates were given a variety of names, including the Minimum Clinically Important Difference, Minimum Important Difference, Minimum Worthwhile Reductions, and Minimum Important Change. Most estimates were obtained using anchor- or distribution-based methods. These methods are not (or not directly) based on patients’ perceptions, are not intervention-specific, and are not formulated in terms of differences in outcomes with and without intervention.Conclusion: The methods used to estimate the smallest worthwhile effect of interventions for low back pain have important limitations. We recommend that the benefit–harm trade-off method be used to estimate the smallest worthwhile effects of intervention because it overcomes these limitations.

Specific instructions for estimating unclearly reported blinding status in randomized trials were reliable and valid

2011-12-26

Abstract: Objective: To test the reliability and validity of specific instructions to classify blinding, when unclearly reported in randomized trials, as “probably done” or “probably not done.”Study Design and Setting: We assessed blinding of patients, health care providers, data collectors, outcome adjudicators, and data analysts in 233 randomized trials in duplicate and independently using detailed instructions. The response options were “definitely yes,” “probably yes,” “probably no,” and “definitely no.” We contacted authors for data verification (46% response). For each of the five questions, we assessed reliability by calculating the agreement between the two reviewers and validity by calculating the agreement between reviewers’ consensus and verified data.Results: The percentage with unclear blinding status varied between 48.5% (patients) and 84.1% (data analysts). Reliability was moderate for blinding of outcome adjudicators (κ=0.52) and data analysts (κ=0.42) and substantial for blinding of patients (κ=0.71), providers (κ=0.68), and data collectors (κ=0.65). The raw agreement between the consensus record and the author-verified record varied from 84.1% (blinding of data analysts) to 100% (blinding of health care providers).Conclusion: With the possible exception of blinding of data analysts, use of “probably yes” and “probably no” instead of “unclear” may enhance the assessment of blinding in trials.

“Might” or “suggest”? No wording approach was clearly superior in conveying the strength of recommendation

2011-11-11

Abstract: Objective: To compare different wording approaches for conveying the strength of health care recommendations.Study Design and Setting: Participants were medical residents in Canada and the United States. We randomized them to one of three wording approaches, each expressing two strengths of recommendation, strong and weak: (1) “we recommend,” “we suggest;” (2) “clinicians should,” “clinicians might;” (3) “we recommend,” “we conditionally recommend.” Each participant received one strong and one weak recommendation. For each recommendation, they chose a hypothetical course of action; we judged whether their choice was appropriate for the strength of the recommendation.Results: The response rate was 77% (341/441). Most participants, in response to strong recommendations, chose hypothetical courses of action appropriate for weak recommendations. None of the wording approaches was clearly superior in conveying the strength of a recommendation. However, different approaches appeared superior depending on the strength and direction (for or against an intervention) of the recommendation.Conclusion: No wording approach was clearly superior in conveying the strength of recommendation. Guideline developers need to make the connection between the wording and their intended strength explicit.

Normative arguments and new solutions for the unbiased registration and publication of clinical trials

Daniel Strech — 2011-10-20

Abstract: Objective: To present a structured account of ethical problems and possible solutions related to selective publication and incomplete trial registration.Study Design and Setting: The presentation of ethical problems and possible solutions is structured using the tools of conceptual normative analysis.Results: Selective publication runs contrary to (1) principles of ethical research, such as social value and respect for participants, (2) sound medical decision making and clinical guideline development, (3) appropriate patient information, (4) public trust in clinical research, and (5) just allocation of public resources for clinical research. Reasons against the obligation of complete registration and publication of trials can be divided into (1) protection of private data and (2) commercial interests. Empirical findings indicate that selective publication and incomplete trial registration (1) are frequent, (2) extensively distort patient-relevant outcomes, and (3) affect a large number of patients.Conclusion: Empirical data and normative arguments outweigh their counterarguments and present a clear case in favor of an even more restrictive obligation to register trials. Institutional review boards and better-educated stakeholders might play a crucial role in facilitating unbiased registration and publication of clinical research. For evaluation purposes, the field needs better standards for study protocols.

A capture-recapture analysis demonstrated that randomized controlled trials evaluating the impact of diagnostic tests on patient outcomes are rare

2011-10-17

Abstract: Objective: To estimate the number of randomized controlled trials (RCTs) published annually that evaluate the impact of diagnostic tests on patient outcomes to gauge the extent of available randomized evidence assessing the effectiveness of diagnostic tests.Study Design and Setting: Relevant RCTs published in 2004–2007 were identified from electronic searches of the Cochrane Central Register of Controlled Trials (CENTRAL). Two search strategies were developed, one using diagnostic methodological terms and one using test names. Potentially relevant RCTs were identified by screening titles and abstracts. Final inclusion decisions were based on full-text review. A random 10% sample of all citations was independently screened by a second reviewer. Capture-recapture methodology was used to estimate the number of relevant RCTs missed by both searches.Results: One hundred thirty-five relevant RCTs were identified from the 23,888 records retrieved. Interobserver agreement was substantial. Capture-recapture methodology estimated that 148 (95% confidence interval: 140, 160) relevant RCTs were published in the 4-year period, an average of only 37 publications per year.Conclusion: RCTs of diagnostic tests that evaluate patient outcomes are rare. Consequently recommendations on the use of diagnostic tests can rarely be made on the basis of randomized comparisons, lower grade evidence frequently being the best available.

Alternatives for randomization in lifestyle intervention studies in cancer patients were not better than conventional randomization

2011-08-19

Abstract: Objective: Assessing effects of lifestyle interventions in cancer patients has some specific challenges. Although randomization is urgently needed for evidence-based knowledge, sometimes it is difficult to apply conventional randomization (i.e., consent preceding randomization and intervention) in daily settings. Randomization before seeking consent was proposed by Zelen, and additional modifications were proposed since. We discuss four alternatives for conventional randomization: single and double randomized consent design, two-stage randomized consent design, and the design with consent to postponed information.Study Design and Setting: We considered these designs when designing a study to assess the impact of physical activity on cancer-related fatigue and quality of life. We tested the modified Zelen design with consent to postponed information in a pilot. The design was chosen to prevent drop out of participants in the control group because of disappointment about the allocation.Results: The result was a low overall participation rate most likely because of perceived lack of information by eligible patients and a relatively high dropout in the intervention group.Conclusion: We conclude that the alternatives were not better than conventional randomization.

Three principles to define the success of a diagnostic study could be identified

Werner Vach, Oke Gerke, Poul Flemming Høilund-Carlsen — 2011-10-14

Abstract: Objective: Diagnostic studies are typically studies with two endpoints, sensitivity and specificity. To define the success of a diagnostic study, results for these two endpoints have to be combined in an appropriate manner.Study Design and Setting: Identification of criteria to define the success of a diagnostic study on a single binary test and investigation of common statistical approaches in relation to these criteria.Results: Three criteria for defining the overall success of a diagnostic study could be identified: a strong criterion, a liberal criterion, and a weak criterion. The strong criterion can be implemented by comparing the lower bounds of the confidence intervals for sensitivity and specificity with prespecified target values, as is typically done in many diagnostic studies. The liberal criterion allows a clinically meaningful compensation between sensitivity and specificity and can be implemented in different ways. If the liberal criterion is applied instead of the strong criterion, this can lead to a substantial reduction in the sample size required for a diagnostic study. The weak criterion is not very adequate for defining the success of a diagnostic study.Conclusion: When planning and analyzing diagnostic studies, the criterion to define the success of the study should be clearly prespecified. The results of the statistical approach taken should be interpreted in accordance with this criterion. This ensures coherence of results and prevents unnecessarily large sample sizes. The liberal criterion should be paid more attention to in the future.

The size of a pilot study for a clinical trial should be calculated in relation to considerations of precision and efficiency

Julius Sim, Martyn Lewis — 2011-12-12

Abstract: Objective: To investigate methods to determine the size of a pilot study to inform a power calculation for a randomized controlled trial (RCT) using an interval/ratio outcome measure.Study Design: Calculations based on confidence intervals (CIs) for the sample standard deviation (SD).Results: Based on CIs for the sample SD, methods are demonstrated whereby (1) the observed SD can be adjusted to secure the desired level of statistical power in the main study with a specified level of confidence; (2) the sample for the main study, if calculated using the observed SD, can be adjusted, again to obtain the desired level of statistical power in the main study; (3) the power of the main study can be calculated for the situation in which the SD in the pilot study proves to be an underestimate of the true SD; and (4) an “efficient” pilot size can be determined to minimize the combined size of the pilot and main RCT.Conclusion: Trialists should calculate the appropriate size of a pilot study, just as they should the size of the main RCT, taking into account the twin needs to demonstrate efficiency in terms of recruitment and to produce precise estimates of treatment effect.

Validation study of cause of death statistics in Cape Town, South Africa, found poor agreement

2011-12-12

Abstract: Objective: The validity of the underlying cause of death on death notification forms was assessed by comparing it to the underlying cause determined independently from medical records.Study Design and Setting: Retrospective study of 703 deaths in two suburbs of Cape Town, South Africa. Two medical doctors completed a medical review death certificate to validate the registration death certificate for each decedent. Agreement, sensitivity, and positive predictive value were measured for underlying causes of death using the World Health Organization (WHO) mortality tabulation list 1.Results: Agreement was poor, with only 55.3% (95% confidence interval [CI]: 51.7, 59.0) of diagnoses matching at WHO mortality tabulation list 1 level. Validity of reported causes of death was poor for HIV, cardiovascular diseases, and diabetes. With correct reporting, the cause-specific mortality fraction for HIV increased from 11.9% to 18.3% (53.6%; 95% CI: 36.9, 77.6), for ischemic heart disease from 3.3% to 7.3% (121.7%; 95% CI: 53.5, 228.7), and for hypertensive diseases from 3.3% to 5.7% (73.9%; 95% CI: 14.4, 167.8). For diabetes, the mortality fraction decreased from 6.0% to 2.3% (−64.3%; 95% CI: −77.1, −37.8) and for ill-defined deaths from 7.4% to 2.3% (−69.2%; 95% CI: −81.0, −51.6).Conclusion: Current cause-specific mortality levels should be cautiously interpreted. Death certification training is required to improve the validity of mortality data.

Combining longitudinal studies showed prevalence of disease differed throughout older adulthood

Allison A.M. Bielak, Julie E. Byles, Mary A. Luszcz, Kaarin J. Anstey — 2011-11-10

Abstract: Objectives: Disease prevalence rates are often generalized across the older adult age range. By pooling self-reported health data from five Australian longitudinal studies of aging, we were able to present disease prevalence rates by 5-year age bands and sex. We also investigated the influence of education on prevalence at each age range and compared our observed prevalence rates with those from the 2001 National Health Survey (NHS) to see if existing data could be used to augment national estimates.Study Design and Setting: We used data on 12,718 adults between 60 and 105 years of age from the Dynamic Analyses to Optimise Ageing (DYNOPTA) project.Results: Hypertension and arthritis were the most prevalent diseases, with approximately 30% of males and 45% of females having either condition. Nearly all diseases were most prevalent amongst older adults in their 70s and lower for individuals in their 60s, and 80s and older. The effect of education varied by disease and older age group. Prevalence rates from DYNOPTA were generally similar to those reported by the NHS.Conclusion: Disease prevalence is not consistent across older adulthood. Combining longitudinal studies provided a sufficient sample to estimate precise age divisions and can be used to supplement national estimates for specific populations.

Using item response theory improved responsiveness of patient-reported outcomes measures in carpal tunnel syndrome

Per-Erik Lyrén, Isam Atroshi — 2011-12-16

Abstract: Objective: To compare responsiveness based on item response theory (IRT) with that based on conventional scoring for two patient-reported outcomes measures in carpal tunnel syndrome (CTS); the short disabilities of the arm, shoulder, and hand (QuickDASH) measure, and the 6-item CTS symptoms scale (CTS-6).Study Design and Setting: Prospective cohort study of patients with CTS undergoing carpal tunnel release at one orthopedic department. Of 455 consecutive patients, 343 completed the QuickDASH and the CTS-6 before and within 1 year after surgery. IRT-based and conventional scores were compared in subgroups according to global rating of change in hand status and treatment satisfaction. The effect size (ES) and the area under the receiver operating characteristic (ROC) curve were used as measures of responsiveness.Results: The mean value for the IRT-based QuickDASH estimate was −0.09 (standard deviation [SD]=1.13) preoperatively and −2.14 (SD=1.79) postoperatively (ES=−1.8) and for the CTS-6 estimate was 0.29 (SD=1.36) preoperatively and −3.87 (SD=2.3) postoperatively (ES=−3.1), indicating very large improvement. The ES for the QuickDASH and CTS-6 were very large (−2.4 and −3.8), respectively, in the group with the largest perceived improvement and decreased with lower perceived improvement. The ES was consistently larger with IRT-based scoring than conventional scoring. The AUC for the QuickDASH and CTS-6 exceeded 0.85.Conclusion: IRT-based scoring showed high responsiveness for the QuickDASH and CTS-6, and the ES were larger than those estimated using conventional scoring.

Risk predictions for individual patients from logistic regression were visualized with bar–line charts

Jonas Björk, Ulf Ekelund, Mattias Ohlsson — 2011-11-23

Abstract: Objective: The interface of a computerized decision support system is crucial for its acceptance among end users. We demonstrate how combined bar–line charts can be used to visualize predictions for individual patients from logistic regression models.Study Design and Setting: Data from a previous diagnostic study aiming at predicting the immediate risk of acute coronary syndrome (ACS) among 634 patients presenting to an emergency department with chest pain were used. Risk predictions from the logistic regression model were presented for four hypothetical patients in bar–line charts with bars representing empirical Bayes adjusted likelihood ratios (LRs) and the line representing the estimated probability of ACS, sequentially updated from left to right after assessment of each risk factor.Results: Two patients had similar low risk for ACS but quite different risk profiles according to the bar–line charts. Such differences in risk profiles could not be detected from the estimated ACS risk alone. The bar–line charts also highlighted important but counteracted risk factors in cases where the overall LR was less informative (close to one).Conclusion: The proposed graphical technique conveys additional information from the logistic model that can be important for correct diagnosis and classification of patients and appropriate medical management.

Tradeoffs between accuracy measures for electronic health care data algorithms

Jessica Chubak, Gaia Pocobelli, Noel S. Weiss — 2011-12-26

Abstract: Objective: We review the uses of electronic health care data algorithms, measures of their accuracy, and reasons for prioritizing one measure of accuracy over another.Study Design and Setting: We use real studies to illustrate the variety of uses of automated health care data in epidemiologic and health services research. Hypothetical examples show the impact of different types of misclassification when algorithms are used to ascertain exposure and outcome.Results: High algorithm sensitivity is important for reducing the costs and burdens associated with the use of a more accurate measurement tool, for enhancing study inclusiveness, and for ascertaining common exposures. High specificity is important for classifying outcomes. High positive predictive value is important for identifying a cohort of persons with a condition of interest but that need not be representative of or include everyone with that condition. Finally, a high negative predictive value is important for reducing the likelihood that study subjects have an exclusionary condition.Conclusion: Epidemiologists must often prioritize one measure of accuracy over another when generating an algorithm for use in their study. We recommend researchers publish all tested algorithms—including those without acceptable accuracy levels—to help future studies refine and apply algorithms that are well suited to their objectives.

Qualitative research—specialized or fragmented?

Joanna E.M. Sale — 2012-03-01

In the article “How different is qualitative health research from qualitative research? Do we have a subdiscipline?,” Morse proposes that qualitative health research conducted by clinicians constitutes a subdiscipline within qualitative research. The author refers to qualitative researchers who do not have medical/health professional education and licensure as “outsiders” who are “fascinated by medical/health topics,” “ignorant of regulations,” and who may “find the critically ill frightening,” “be haunted by sounds of distress,” and “not know (the) health/medical literature.” She further argues that a qualitative researcher without a nursing or medical background often cannot recognize the patient’s condition and signs of fatigue and thus pace their data collection strategies accordingly. Implied in her article are the ideas that qualitative researchers who are not health professionals are not “street smart,” cannot recognize appropriate research questions, and cannot more readily make realistic recommendations for practice.

Erratum to: “CONSORT 2010 Explanation and Elaboration: updated guidelines for reporting parallel group randomised trials” [J Clin Epidemiol 2010;63(8):e1–37]

2011-11-07

In this Research Methods & Reporting article by David Moher and colleagues a minor error occurred in Table 3. In the fourth row (“Treatment allocation”), the text in the second and third cells should be “Participants who received [not ‘completed’] treatment as allocated, by study group” and “Participants who did not receive [not ‘complete’] treatment as allocated, by study group,” respectively.