A convolutional network trained on dermoscopic images can match or exceed the sensitivity of board-certified dermatologists at telling melanoma from nevus, provided the test set comes from the same distribution as the training data. By mid-2019 that is a settled result. The hard part is not obtaining that number on a benchmark: it is working out how much of it survives once the image arrives from a different sensor, under different lighting, inside a laboratory workflow I did not build around the model.

Context

The reference publication remains Esteva et al. (Nature, 2017): an InceptionV3 pre-trained on ImageNet and fine-tuned on roughly 129,450 clinical images, set against 21 board-certified dermatologists on carcinoma versus benign lesion and melanoma versus nevus. The model matches the human panel on the area under the ROC (Receiver Operating Characteristic) curve. Since then the literature has replicated and in places exceeded the human figure on narrow tasks: Brinker et al. (European Journal of Cancer, 2019) describe a convolutional network (CNN) that beats 136 of 157 dermatologists in a direct dermoscopic classification, with p < 0.001 against the panel.

All of these results share one assumption: the test set is dermoscopic, curated, biopsy-labelled, and distributed like the training set. Public datasets made these benchmarks reproducible — the International Skin Imaging Collaboration (ISIC) archive, and in particular HAM10000 (Tschandl et al., Scientific Data, 2018; 10,015 images across seven diagnostic categories). It is a good starting point and a poor measure of what happens in production.

Architecture

For classification, the dominant approach in mid-2019 is transfer learning on an ImageNet backbone: InceptionV3, ResNet, or DenseNet where depth at a given parameter count is needed. The lower layers are frozen, the upper ones fine-tuned, the head replaced with the dermatological classes. End-to-end training from scratch, as in Esteva, asks for orders of magnitude more labelled images than a single laboratory holds.

For lesion segmentation — separating lesion pixels from healthy-skin pixels — the reference is U-Net (Ronneberger, Fischer, Brox, MICCAI 2015): a contracting path for context, a symmetric expanding path for localisation, skip connections that recover the spatial resolution lost in downsampling. The standard testbed is task 1 of ISIC 2018 (2,594 training images, binary masks). Segmentation comes before classification when a lesion-centred crop and an area measurement are needed, but it brings along a second model to train, validate, and version.

Inference is not the bottleneck: a U-Net segments a 512×512 image in under a second on a GPU, and an InceptionV3 classifies in tens of milliseconds. The cost lies elsewhere.

Critical point

Between the published benchmark number and the laboratory number sits distribution shift. Three causes, in order of severity as I have seen them:

  • Acquisition domain. Models trained on dermoscopy get worse on clinical camera images, and the reverse holds. Scale, focus, and the presence of dermoscopic artefacts (bubbles, hairs, gel reflections) all change. A model that has never seen a hair cross the lesion mistakes it for an edge.
  • Annotation bias. In HAM10000 over half the labels are confirmed by histopathology, the rest by follow-up, expert consensus, or confocal microscopy. The categories are imbalanced: melanocytic nevi dominate. A network maximising raw accuracy learns to answer “nevus” and is nearly always right. The metric that counts is not accuracy but melanoma sensitivity at a fixed specificity.
  • Spurious markers. Networks learn correlations present in the dataset but not causal: adhesive rulers, ink skin markers, and dressings appear more often beside lesions a clinician has already judged suspicious. The model learns the ruler, not the lesion.

None of these is fixed by more layers or more epochs. They are fixed by data covering the real acquisition domain and by validation on a test set collected separately, ideally at another centre.

Integration

An image pipeline that returns a score is not yet a system usable in a laboratory. Between the model and the report there are constraints with nothing to do with deep learning, and those are what decide whether the model is ever used.

The first is sample traceability. A dermatological image must be bound unambiguously to patient, accession, acquisition device, operator, and timestamp. In laboratory workflows this runs over HL7 (Health Level Seven) v2.x messaging — ORM messages for orders, ORU for results — to the Laboratory Information System (LIS). The score of a model that does not attach to a traceable accession is an orphan number: it cannot be reported, cannot be verified, cannot be defended.

The second is version reproducibility. A score produced today must stay reconstructible: which model, which weights, which pre-processing, which threshold. Without it you cannot re-evaluate a result months later, when the histology arrives — and that comparison is the most valuable clinical-validation datum you have.

The third is placement in the decision flow. A classifier with high sensitivity and moderate specificity serves as a triage or second-read tool, not as an autonomous decider. The operating threshold is set against the clinical cost of false negatives, and that cost is defined by the clinician, not by the ROC curve.

Limits

In mid-2019 the solid result is binary classification on curated dermoscopic images, distributed like the training set. Outside that the guarantees give way quickly: multiclass classification across many rare diagnoses is held back by the scarcity of examples per class; generalisation across different datasets and devices is fragile; prospective validation on real clinical workflows is largely still to be done, and the cited work says so openly.

The metrics reported here hold for the public test sets. On a real laboratory workflow they must be re-measured on that workflow’s own data, because that is the only place where its distribution shift becomes observable. The model is the part that transfers; the validation is not.

The ALA project, which brings these networks into a real clinical diagnostic workflow, is described in an insight published by noze: https://www.noze.it/en/insights/ala-project/.

Cover image: Dermatoscopic view of a malignant skin melanoma: an irregularly bordered dark brown-black pigmented lesion on skin, with markings… — photo by Dermanonymous, CC BY-SA 4.0 — https://commons.wikimedia.org/wiki/File:Malignant_Melanoma_Left_Mid_Back_Dermatoscope.jpg