When you segment a new medical dataset, result quality is decided by how preprocessing, topology and training adapt to the data, not by the network architecture. That is the claim of the preprint nnU-Net: Self-adapting Framework for U-Net-Based Medical Image Segmentation (Isensee et al., arXiv 1809.10486, deposited 27 September 2018), out of Klaus H. Maier-Hein’s group at the DKFZ in Heidelberg.

Context

The U-Net of Ronneberger, Fischer and Brox (MICCAI 2015) became within a few years the de facto reference for biomedical segmentation: a contracting-path encoder, an expanding-path decoder, skip connections between the two. The literature has since accumulated variants — residual encoders, dense blocks, attention mechanisms, 3D versions, alternative losses — each with its own claimed gains on a subset of tasks.

The practical problem is transferability. A configuration that runs well on a multimodal brain MRI does not run on a liver CT with anisotropic voxels, and the reverse holds too. Architecture, normalisation scheme, patch size, augmentation, scheduler, inference strategy: there are many degrees of freedom, and they are coupled to one another. Anyone setting up a segmentation pipeline spends most of the time hand-tuning these choices, and the tuning does not carry over to the next dataset.

Problem

The preprint restates the question this way: how much of the gain claimed by architectural variants survives once the baseline U-Net is configured well? For the authors, the pipeline decisions weigh more — resampling spacing, modality-appropriate normalisation, patch size bounded by GPU memory, network depth derived from the patch size — than the architectural additions. Hence the name: no-new-Net. The object of the work is not a new network but the system that configures it.

Architecture

nnU-Net starts from an analysis of the dataset, what the authors call the dataset fingerprint: voxel spacing, distribution of volume sizes, intensity distribution, number of classes and their frequency. From this profile a set of rules derives the pipeline parameters, without trial-and-error search on the individual task.

The derived decisions include:

  • Resampling: a target spacing, with separate handling of the lowest-resolution axis in anisotropic volumes.
  • Intensity normalisation: for CT, percentile clipping and standardisation over the Hounsfield Unit (HU) values collected from the annotated regions; for MRI, per-volume z-score normalisation.
  • Patch size and topology: the patch size is the largest the memory allows, and the number of pooling operations follows from the patch size, so that the receptive field covers the whole patch.
  • Loss and schedule: a sum of Dice and cross-entropy, extended training with a polynomial learning-rate decay.

From these rules nnU-Net generates up to three configurations for the same dataset:

  • 2D, a U-Net trained slice by slice;
  • 3D full resolution, a 3D U-Net on the volume at working resolution;
  • 3D cascade, with a first low-resolution network producing a coarse segmentation and a second full-resolution network refining it — useful when memory is too tight to fit the organ’s context into a single patch.

The final configuration is chosen by 5-fold cross-validation, possibly as an ensemble of several configurations. What stays constant is the standard U-Net: no attention blocks, no residual variant, no architectural tricks.

The critical point

The testbed is the Medical Segmentation Decathlon, a challenge associated with MICCAI 2018 spanning ten heterogeneous tasks — brain tumour, heart, liver, hippocampus, prostate, lung, pancreas, hepatic vessels, spleen, colon — across different modalities and geometries, with no manual adjustments between datasets. The challenge is built to reward generalisation, not optimisation on a single problem.

The preprint’s abstract reports that, at the time of manuscript submission, nnU-Net achieves the highest mean Dice scores across all classes and seven phase-1 tasks on the challenge’s online leaderboard, the sole exception being class 1 of the BrainTumour task. This is a provisional result, referring to a public leaderboard open to new submissions: it should be read as the state of an ongoing competition, not as a settled outcome. It remains notable that what obtains it is a U-Net with no architectural changes, configured automatically.

Implications

If the claim holds up under review, an operational criterion follows for anyone working on medical segmentation: before comparing a new method against the literature, compare it against a competently configured standard U-Net. A good share of the gap attributed to architecture may be a configuration gap.

For clinical groups the consequence is the entry threshold. The pipeline adapts on its own to the structure of the annotated dataset; anyone wanting to train a segmenter on their own data — tumour volumetry, organs at risk for radiotherapy — need not write and tune training code from scratch. It remains to be verified how well the fingerprint holds on heterogeneous datasets, collected with different scanners and protocols.

Limits

The preprint does not describe a zero-cost solution. The 5-fold cross-validation on 3D volumes takes training times on the order of days on a single GPU, and generating several configurations multiplies the compute. The fingerprint presupposes a relatively coherent dataset: uniform scanners, protocols and population; how far the automatic derivation degrades on very heterogeneous collections remains an open empirical question. The pipeline is optimised for result quality in research, not for the fast inference that clinical production would need. And the architectural space explored remains that of the U-Net: it is taken for granted that, for segmentation, configuring a U-Net well is enough — an assumption the Decathlon tests but does not close.

The code is announced as open source on GitHub (MIC-DKFZ/nnUNet); the real test will come when external groups reproduce the results on their own data and the later phases of the challenge close.

https://arxiv.org/abs/1809.10486 https://link.springer.com/chapter/10.1007/978-3-319-24574-4_28 https://github.com/MIC-DKFZ/nnUNet https://www.noze.it/en/insights/nnunet-framework-segmentazione/

Cover image: Axial T2-weighted brain MRI scan showing a bright (hyperintense) tumoral lesion in the cortex of one hemisphere — photo by Manoj Kumar, Raghu H. Ramakrishnaiah, Rohan Samant, CC BY-SA 4.0 — https://commons.wikimedia.org/wiki/File:4-year-old-boy-with-angiocentric-glioma-Axial-T2weighted-image-shows-a-hyperintense.png