To train a segmentation network on 3D medical images the bottleneck is labels: every volume has to be segmented voxel by voxel by a qualified operator, and across a dataset of hundreds of cases that work weighs more than the network architecture or the GPU. MONAI Label, an open-source sub-project of Project MONAI (Apache 2.0), attacks exactly that cost: it turns annotation from manual drawing into the correction of a proposal the model generates.

Context

In the traditional flow the expert annotates N cases from scratch, a model is trained, then evaluated. The annotation cost is fixed and comes entirely before any return from the model. MONAI Label flips the order: a first model proposes a segmentation, the operator corrects it, and the corrections feed an incremental retraining. The better the model gets, the fewer voxels are left to correct. The reference paper (Diaz-Pinto et al., arXiv:2203.12362, March 2022) measures, on spleen segmentation from the Medical Segmentation Decathlon, per-volume times of 1–2.5 minutes with assisted annotation against roughly 25 minutes for advanced manual contouring, and a 50–80% drop in the time to annotate 220 lesions in a neurosurgery case.

These are figures measured on specific tasks, not general guarantees: the gain depends on how well the initial model fits the domain. They do point to where the saving concentrates — on the easy cases, already nearly correct — and to the practical consequence: annotation becomes an effort that shrinks over time rather than a constant cost.

Architecture

MONAI Label is client-server. The server is a Python application that exposes a REST API and loads one or more apps, each gathering the tasks of a pipeline. An app inherits the MONAILabelApp interface and declares its tasks in distinct categories:

  • infer — invokes a model (pre-trained or updated by local training) and produces a segmentation when the client asks for one;
  • train — retrains the model in the background on the collected annotations;
  • strategy — picks which unlabelled volume to propose first, following an acquisition function;
  • scoring — computes per-volume metrics, including the uncertainty estimates the strategy relies on.

The clients are not an interface of their own: they are plugins for viewers that already exist, so the radiologist keeps working in the tool they know. The paper describes two frontends, 3D Slicer (desktop) and OHIF (browser); the latter talks to archives over DICOMweb, where the server uses WADO-RS for retrieval, STOW-RS for storage and QIDO-RS to query collections. Version 0.6.0 (December 2022) added pathology models and an extension for QuPath, carrying the same scheme to whole slide images in digital pathology. Today 0.6.0 is the latest stable release.

Three interaction methods are exposed. DeepGrow guides segmentation with positive and negative clicks, in 2D slice by slice or across the whole 3D volume. DeepEdit produces a first automatic segmentation with no clicks, then accepts point corrections: its training mixes simulated clicks with non-interactive training, so the same model covers both auto-segmentation and refinement. The scribbles method lets the user draw free foreground and background strokes and completes the mask through an energy optimisation, as an alternative to the network prediction or downstream of it.

Critical point

What separates MONAI Label from a plain inference service is the link between model uncertainty and annotation order. The strategy does not present cases at random: it uses the uncertainty estimates to prioritise the volumes the model is least sure about, because correcting those cases teaches the retraining more than confirming cases that are already easy.

The framework separates two kinds of uncertainty. Aleatoric — the noise intrinsic to the data — is estimated with Test Time Augmentation, applying transforms to the input and measuring the variance of the predictions. Epistemic — the model’s ignorance, which falls with more data — is estimated with Monte Carlo Dropout, keeping dropout active at inference and running several passes. It is the second that drives active learning: a volume with high epistemic uncertainty flags a region of the data space the training set covers poorly, hence a candidate where a human label pays off most.

It is also the part that demands discipline. Uncertainty estimates depend on how well the model is calibrated: on a poorly trained or badly calibrated network they order the cases worse than chance. And every incremental retraining yields a new model: without versioning the states and tracking which annotations produced which checkpoint, the reproducibility of the loop vanishes.

Implications

The architecture does not tie down the deployment: the same REST API runs on a single GPU workstation, on an on-premise GPU server shared by several radiologists, or in a container on a cluster. In healthcare that makes the difference, because data can rarely leave the hospital perimeter: the server can sit next to the PACS (Picture Archiving and Communication System) and the clients connect over the local network, with no imaging moved to third parties.

For teams building datasets — multicentre trials, anatomical atlases, organ-at-risk contouring in radiotherapy planning — the value is in the loop: start from a generic model, specialise it on your own domain by correcting the cases the strategy proposes, and measure quality along the way instead of discovering it only at the end of annotation.

Limits

A generic model shipped with MONAI Label usually stays below a custom model trained on local data; the realistic scenario still includes a fine-tuning phase, and the advantage of assisted annotation shows only once the initial model is already good enough to generate proposals that can be corrected quickly. On a structure it has never seen, the first cases cost almost as much as manual annotation.

Then there are the operational themes: governing model versioning and the reproducibility of incremental training, documentation quality (broad but fragmented, with a steeper learning curve than a commercial tool with dedicated support), and the fact that MONAI Label is a research and data-preparation tool. It carries no regulatory marking: turning its output into a certified medical device requires the usual qualification work under MDR (Medical Device Regulation), which stays outside the framework.

Cover image: 3D volume rendering of a human head in 3D Slicer: skull and cerebral vessels shown semi-transparent, with a clipping plane revealing… — illustration by 3D Slicer project, CC BY-SA 3.0 — https://commons.wikimedia.org/wiki/File:3DSlicer-VolumeRendering1.png