AI-Based Model Outperforms Teledermatologists in Diagnosing Lesions of Concern

Diagnostic accuracy for possible skin cancer lesions may be improved by an automated image analysis program.

An automated image analysis program may improve dermatologist diagnostic accuracy for lesions of concern, data from a research letter published in the Journal of the American Academy of Dermatology indicates.

Australia currently has a shortage of dermatologists, resulting in long wait times for patients. In an attempt to improve patient access to care, MoleMap Ltd and Monash eResearch developed a convolutional neural network (CNN)-based algorithm that automatically classified lesions of concern from images.

To test the CNN algorithm, the prospective clinical trial, improving Skin cancer Management with ARTificial Intelligence (SMARTI), recruited patients (N=214) with 1 or more skin lesions of concern (n=743) from the Alfred Hospital and Skin Health Institute in Australia from 2019 through 2021. Lesions were diagnosed and a management plan was formulated by a dermatology resident and a dermatologist. Images of the lesions were then graded by the CNN and a teledermatologist. The performance of the CNN model was compared with the teledermatologist’s and treating dermatologist’s opinions.

The CNN model had an area under the receiver operating characteristic curve (AUC) of 0.837 for diagnosing lesions, compared with an AUC of 0.807 for the teledermatologist’s (P =.050) diagnosis. The in-person treating resident had an AUC of 0.847 and with the CNN outcome, the accuracy improved to an AUC of 0.879 (P =.009). That is, “the CNN had a positive impact on resident management decisions, shifting them to be more in line with those of the treating dermatologist.”

The CNN had a positive impact on resident management decisions, shifting them to be more in line with those of the treating dermatologist.

Overall, the CNN falsely identified 3 actinic keratoses (AKs), 2 metastatic deposits of melanoma (MDM), and 1 melanoma in situ (MIS), for a false negative rate of 3.7%. The teledermatologists falsely identified 6 MIS, 4 AKs, 2 MDM, 2 intraepidermal carcinomas, and 1 basal cell carcinoma, for a false negative rate of 9.3%. The false negative rate for the in-person consultation with the treating dermatologists was 0.6%, in which one MIS was falsely identified.

The CNN did, however, lead to unnecessary biopsy in some patients. The treating dermatologists recommended that 7 lesions should be monitored, but, after consulting with the CNN algorithm, these 7 lesions were biopsied and ultimately found to be benign. The patients indicated that they preferred having a biopsy rather than long-term monitoring.

The major limitation of this study was the convenience sampling, which may have introduced sampling bias.

The results of this study suggest to investigators that an automated CNN-based model may have the potential to positively affect dermatological decisions and could address some of the unmet needs due to a dermatologist shortage in Australia. They continued to write that this model, however, requires additional research and would likely not be suitable for certain lesions, such as acral or scalp lesions, or for patients with more diverse skin types.

Disclosure: Several authors declared affiliations with industry. Please refer to the original article for a full list of disclosures.

References:

Felmingham C, Pan Y, Kok Y, et al. Improving skin cancer management with ARTificial Intelligence (SMARTI): a pre-post intervention trial of an artificial intelligence system used as a diagnostic aid for skin cancer management in a real-world specialist dermatology setting. J Am Acad Dermatol. 2022;S0190-9622(22)02964-4. doi:10.1016/j.jaad.2022.10.038