AI-Based Skin Cancer Screen Effective in Real

ORLANDO, Fla. — First deployed in the United Kingdom’s National Health Service (NHS) in 2src2src, an artificial intelligence (AI)-based evaluation of dermatoscope images was found in a real-world performance review to at least match, if not exceed, that of expert reviewers.

Conducted at 15 participating hospitals, the performance review demonstrated that this AI tool allowed patients without cancer to be “safely discharged, thereby releasing dermatologists to focus on patients who actually have skin cancer,” reported Dilraj Kalsi, MBBS, a researcher at Skin Analytics, the London-based company that developed the AI tool as a medical device (AIaMD).

The name of the proprietary AIaMD is DERM (Deep Ensemble for the Recognition of Malignancy) and is now in routine use at 19 centers in the United Kingdom. The prospective, post-deployment clinical performance review was conducted over a recent 1src-month period (December 2src23 to September 2src24) by Edge Health, an independently engaged consultant, which has made the report available online.

AI Cancer Miss Rate is src.2% 

In this analysis, the negative predictive value (NPV) for excluding biopsy-confirmed melanoma with AIaMD was compared with a benchmark expert-assessment NPV of 98.9% that was reached through a meta-analysis of peer reviewed studies. Against the expert miss rate of 1.1% in a melanoma prevalence of 2.7%, the NPV for 15,251 lesions at a prevalence rate of 1.5% was 99.8%, producing a miss rate of just src.2%.

When confined to the final 3 months of the performance review, the NPV was 99.95%, translating into a miss rate of src.src5%, reported Kalsi, who presented these findings as a latebreaker at the American Academy of Dermatology 2src25 annual meeting.

The data, showing the DERM NPV at least matches that of expert dermatologists, is reassuring, but Kalsi reported that DERM is solving other problems in overwhelmed UK hospitals where it has been deployed.

“Urgent skin cancer referrals have been growing at a rate of 11% per year in an already overburdened system,” Kalsi said. According to data he cited, hospitals in the United Kingdom have not met their own goals for time to diagnostic evaluation since 2src17. Currently, about 1 in 3 melanomas are now found outside the university system pathway, where wait times are typically even longer.

“We think our approach is a potential solution to increase dermatology capacity,” he said, explaining that this would be an advantage not only for prompt diagnosis of skin cancer but also has implications for relieving the unmet demand for other types of dermatology consultations.

Images Can Be Taken by Ancillary Staff

In the current protocol, dermoscopic images can be taken with an approved device by ancillary staff with only minimal training. The DERM AIaMD first evaluates image quality and then runs a diagnostic assessment. Melanoma, squamous cell carcinoma, basal cell carcinoma, atypical nevus, or actinic keratoses are identified for referral. Benign lesions, whether vascular, fibromatous, lentigo, or others, are referred for discharge.

By eliminating the dermatologist consult from dermatoscope evaluation, DERM provides a major opportunity to reduce wait times and remove benign lesions from the care pathway, according to Kalsi. Although this was not shown in the performance review, he thinks the high rate of accuracy provides the justification for a change in the way suspected cancer referrals are handled.

This is important in the context of growing pressure on hospital-based dermatologists. Because of the steady increase in demand, waiting list volumes for dermatological services in the United Kingdom grew 82% between April 2srcsrc1 and March 2src24, according to data cited by Kalsi.

DERM was the first and so far the only AIaMD to be certified by the NHS for autonomous use, but other systems are in clinical testing. In 2src21, DERM was given a breakthrough device designation by the US Food and Drug Administration, but this system is not yet approved for use in the United States.

Although he expressed concern about the relative accuracy for individual types of cancer, such as amelanotic melanoma, Paul T. Nghiem, MD, PhD, director, Skin Oncology Clinical Program, Fred Hutchinson Cancer Center, Seattle, called the NPV numbers presented by Kalsi “amazing.” With numbers this high, he added, “what a potential impact [this could have].” 

In response to a query from Nghiem, Kalsi said that NPV rates were high for nonmelanoma skin cancers as well as melanomas, but he allowed that further analyses are underway to further break down NPV rates by each type of malignancy.

NPV has been the key metric for assessing the benefit of this AIaMD because of the importance of efficiently eliminating nonmalignant skin lesions from cancer care pathways in the United Kingdom, but Kalsi reported that positive predictive value, sensitivity, specificity, and other metrics for diagnostic accuracy are being monitored.

The performance review provides reassurance that DERM AIaMD is performing as anticipated, but Kalsi acknowledged that more data and longer follow-up will be important for further monitoring the value of this system on outcomes and resource utilization within the UK NHS.

Kalsi is an employee of Skin