Can Computer Algorithms Diagnose Melanoma More Accurately Than Dermatologists?

State-of-the-art computer vision systems have demonstrated accuracy comparable to that of dermatologists in the diagnosis of melanoma from dermoscopy images, according to a recent cross-sectional study published in the Journal of the American Academy of Dermatology.

A total of 100 randomly selected dermoscopic images (50 melanomas, 44 nevi, and 6 lentigines) from an international computer vision melanoma challenge dataset of 379 test images, along with individual algorithm results from 25 teams, were evaluated. The researchers used 5 methods (2 nonlearned approaches and 3 machine learning techniques) to combine the individual automated predictions into “fusion” algorithms. In a comparison analysis, 8 dermatologists classified the lesions in the 100 dermoscopic images as either benign or malignant.

The average sensitivity and specificity of dermatologists for classification of the lesions were 82% and 59%, respectively. At 82% sensitivity, dermatologist specificity was similar to that of the top challenge algorithm (59% vs 62%, respectively; P =.68), but was significantly lower than that of the best-performing fusion algorithm (59% vs 76%, respectively; P =.02). The receiver operating characteristic area of the top fusion algorithm was significantly greater than that of the mean receiver operating characteristic area of the dermatologists (0.86 vs 0.71, respectively; P =.001).

It is important to note that the dataset used in this study lacked the full range of skin lesions commonly encountered in clinical practice, especially banal lesions. Algorithms and readers were not provided with such clinical data as age or lesion history or symptoms. Thus, results obtained with the use of this study design cannot be extrapolated to clinical practice until they are validated in prospective studies.

“Although these results are preliminary and should be viewed with caution, development and comparison of deep learning methods on larger, more varied datasets is likely to accelerate the potential use and adoption of computer vision for melanoma detection,” the researchers concluded.

Reference

Marchetti MA, Codella NCF, Dusza SW, et al; International Skin Imaging Collaboration. Results of the 2016 International Skin Imaging Collaboration International Symposium on Biomedical Imaging challenge: comparison of the accuracy of computer algorithms to dermatologists for the diagnosis of melanoma from dermoscopic images [published online September 29, 2017]. J Am Acad Dermatol. doi: 10.1016/j.jaad.2017.08.016