On 21 April 2020 NVIDIA and King’s College London presented MONAI — Medical Open Network for AI — an Apache 2.0 PyTorch library for deep learning on medical imaging, hosted at github.com/Project-MONAI/MONAI. The news is not a model or an experimental result: it is that the pre-processing, training and evaluation code each group used to rewrite for itself now sits in one place.

Context

People training networks on computed tomography (CT) or magnetic resonance (MR) have worked in PyTorch for years, but around the general-purpose framework sits a layer of code specific to the medical image that nobody had ever standardised. They are always the same operations, repeated in every lab with small variants: resampling voxels to a target spacing, realigning the anatomical orientation, normalising Hounsfield units for CT, extracting patches from volumes too large to fit in memory. On top of that come the losses and metrics needed in radiology — the Dice coefficient, the Hausdorff distance, the average surface distance — implemented case by case, often with different conventions for borders and background voxels.

The result is duplicated code and a more fragile reproducibility. Two papers that report the same pre-processing may have written it in different ways, and without the code it is hard to notice. NVIDIA had already addressed the problem on its own side with Clara Train, part of the Clara stack, but tied to the NVIDIA ecosystem. There were also earlier academic libraries — NiftyNet, DLTK, DeepNeuro — from which MONAI openly states it inherits its practices. What was missing was a neutral component, built directly on top of PyTorch, onto which contributions could converge instead of scattering.

Architecture

MONAI is a Python library on top of PyTorch, with optional PyTorch Ignite integration for the training loop. The split into modules follows the stages of an imaging pipeline.

monai.transforms is the part that pays off the soonest. It gathers the domain transforms: Spacing for resampling to a target voxel spacing, Orientation for realigning to the anatomical axes, ScaleIntensityRange for mapping Hounsfield units, alongside three-dimensional data augmentation (RandAffine, Rand3DElastic, RandGaussianNoise). Each transform exists in two forms: one that works on an array and a dictionary-based variant — by convention with a d suffix — that operates on a {"image": ..., "label": ...} dictionary and applies the same spatial transform consistently to image and segmentation. That is the detail that avoids the commonest mistake: decoupling the mask from the image when you rotate or crop.

monai.data loads the formats you need — NIfTI, DICOM via ITK/SimpleITK, NRRD — and keeps the metadata (spacing, origin, orientation) along the pipeline rather than reducing everything to a bare tensor. monai.networks offers ready architectures, among them 2D/3D U-Net implementations and some residual variants, plus the primitive modules (convolutions, residual units, upsampling) for building new ones. monai.losses collects the segmentation losses — Dice, the Dice-plus-cross-entropy combination, the Focal and Tversky variants for imbalanced classes; monai.metrics holds the evaluation metrics, under the same shared convention.

The recurring idea is composability: the pipeline is a list of transforms that compose, the networks are assembled from primitives, the training loop stays separate from model and data. Nothing conceptually new for anyone who knows PyTorch — the value lies in having these domain-specific primitives already written and in one place.

The critical point

MONAI’s value lies in the shared convention that holds the components together, more than in the list of components itself. The Dice coefficient and the Hausdorff distance are unambiguously defined only as long as everyone agrees on how to treat borders, background voxels and the degenerate case where a class is missing from the ground truth. When each lab writes its own metric, two nominally equal numbers may not be comparable. A shared library makes that convention explicit and inspectable: two groups using DiceMetric are genuinely computing the same thing.

The same holds for the geometric transforms. Spacing and Orientation encode choices — interpolation, reference frame — that otherwise stay implicit and are hardly ever documented in papers. Moving them into a versioned library makes the dependency citable: stating the MONAI version pins down part of the pre-processing in a verifiable way.

The flip side is that the value depends entirely on adoption. A common library used by a single lab is just another library. In the coming months the signal to watch is not the feature set, but how many external groups open pull requests and how many papers state a MONAI version in their methods.

Implications

For anyone building in-house clinical pipelines the sum changes: a slice of the infrastructure work so far repeated in every project becomes an external dependency maintained by several institutions. The Apache 2.0 licence makes it usable inside commercial products and certification-bound workflows too, without the constraints of a proprietary stack — the regulatory conformity of a software medical device stays entirely on the builder’s shoulders, but the layer underneath stops being a single-vendor lock-in.

For research the expected effect is on reproducibility. Known, versioned transforms and training loops narrow the space in which two “identical” implementations quietly diverge. It does not remove the problem — data, hyperparameters and weights weigh far more — but it removes one of its least visible causes.

Limits

At publication MONAI is a young project, at version 0.1, with an API still to consolidate and growing documentation: the 0.x line should be read as a warning that signatures may change. The scope is research — data loading, training, evaluation — not production deployment, which sits outside the initial perimeter. For out-of-the-box segmentation the reference on immediate results remains the self-configuring frameworks such as nnU-Net; MONAI asks for more manual configuration in exchange for reusable components. And standardisation, for now, is only a hypothesis: a library becomes a standard only if the community adopts it, and two months after the announcement it is too early to tell.

https://developer.nvidia.com/blog/nvidia-and-kings-college-london-announce-monai-ai-framework-for-healthcare-research/ https://www.kcl.ac.uk/news/london-ai-centre-and-nvidia-launch-monai-a-new-ai-framework-for-healthcare https://github.com/Project-MONAI/MONAI https://project-monai.github.io/ https://www.noze.it/en/insights/monai-imaging-ai-open/

Cover image: Axial slice of a brain MRI scan with a tumor (glioblastoma) highlighted by a colored segmentation mask overlaid on the lesion — photo by EmmaWiki2020, CC BY-SA 4.0 — https://commons.wikimedia.org/wiki/File:Studierfenster_Brain_Tumor_Segmentation.png