New methods give better estimates of changes in diagnostic accuracy when prior information is provided

Abstract 

Objective

Whether tests such as imaging should be read with or without access to prior clinical information is controversial. Naïve comparisons may suggest that the provision of prior information improves test accuracy, whereas in fact the opposite may be true. This is because provision of clinical background may actually bias test readers to over- or underinterpret relevant test findings, and they may suboptimally integrate the previous and current evidence. We propose comparing the combined accuracy of prior information and a test read (i) with or (ii) without knowledge of prior information. Analysis methods include simple decision rules and logistic regression.

Study Design and Setting

A study of cancer detection in women presenting with breast symptoms, in whom ultrasound could be read with or without reviewing prior mammography.

Results

Naïve analysis gave an area under the receiver operating characteristics curve (AUC) for ultrasound read with mammography on view that was 4.6% higher (P < .01) than without mammography on view. Our approach, comparing the combined accuracy of mammography and ultrasound read i) with and ii) without knowledge of mammographic findings, showed much smaller differences.

Conclusion

Our approach is more appropriate than naïve analyses. The particular choice of analytic method depends on the study size and the diagnostic accuracy of combinations of the prior information and the test reading.

Keywords: Sensitivity and specificity, ROC curve, Diagnostic technique and procedures, Mammography, Ultrasonography, Breast cancer