Since 1999 the National Library of Medicine (NLM) of the National Institutes of Health (NIH) has funded a public library for the segmentation and registration of biomedical images, distributed under a BSD licence: the Insight Toolkit, ITK for short. The three-year contract is coordinated by project manager Terry Yoo and assigned to six contractors — three commercial (GE Corporate R&D, Kitware, MathSoft/Insightful) and three academic (University of North Carolina at Chapel Hill, University of Tennessee, University of Pennsylvania). Public code releases have been out for over a year, and the library is approaching the functional maturity of its first development cycle.

Context

Through the 1990s the quantitative analysis of medical images — segmenting anatomical structures, registering volumes acquired at different times or under different modalities — relied largely on commercial tools (Analyze from the Mayo Clinic, Amira, the MATLAB add-ons) or on laboratory code that was never shared. The same base algorithms — Perona–Malik anisotropic diffusion, Sethian’s fast marching and level sets, Viola and Wells’s mutual information for multimodal registration — were rewritten from scratch across dozens of groups, and from a published paper the results were often impossible to replicate.

The NLM funding grew out of the Visible Human Project, the high-resolution anatomical dataset whose male (1994) and female (1995) volumes were already distributed as a public resource. The contract explicitly aims at a reference library, inspectable and reusable, on which to build algorithms that can be compared with one another, starting from that data.

Architecture

ITK is written in C++ and uses templates heavily. Each filter is parameterised on pixel type and image dimensionality, both resolved at compile time: the same code works on 2D images, 3D images and series of volumes without a single line of rewriting, and the abstraction costs nothing at runtime. It is the philosophy of the Standard Template Library carried into a scientific domain, with one concrete consequence — compile times lengthen, and the errors the compiler prints on templates are famously unreadable.

Processing is organised as a dataflow pipeline. Images pass through chains of filters (itk::ImageToImageFilter and its derivatives); a request for a region of interest travels back along the chain, and intermediate results are recomputed only when an upstream parameter changes. The model comes straight from Kitware’s experience with the Visualization Toolkit (VTK), the scientific-visualisation library the same firm has published as open source since 1996.

Among the base objects are itk::Image, itk::Mesh, itk::PointSet, itk::SpatialObject, and the registration apparatus built around a metric, a transform, an interpolator and an optimiser — interchangeable components that combine to define an alignment experiment.

The build is handled by CMake, a cross-platform generator likewise born at Kitware inside the same contract, whose version 1.0 shipped in November 2001. From a single description CMake produces makefiles on Unix and project files for the Windows development environments: it is what makes ITK compilable on Unix, Windows and Mac, working around the practical limits of autoconf/automake on portable C++.

Algorithms included

The library covers a broad set of reference techniques for medical image computing:

  • Filters — Gaussian, Canny derivative, anisotropic diffusion (Perona–Malik); mathematical morphology (erosion, dilation, opening, closing).
  • Segmentation — region growing, watershed, clustering (K-means, Markov Random Field), level set methods with narrow-band and fast marching schemes.
  • Registration — rigid, affine and deformable (B-spline) transforms; optimisers (gradient descent, Powell, LBFGS); metrics including sum of squared differences, correlation and mutual information.
  • Input/output — the medical imaging formats in use: DICOM, Analyze, MetaImage (.mha), GIPL.

The critical choice

What defines ITK is the use of C++ templating extended across the whole domain. It costs a fair bit — a steep entry barrier, slow compilation, dependence on compilers that handle partial template specialisation well, still a concrete problem in 2002 with some toolchains. In exchange it gives two properties that prove decisive for a computation library: dimensional genericity (a single RegionGrowingFilter serves both 2D and 3D) and no runtime dispatch on numeric types, which on volumes of hundreds of megabytes is no detail.

There is a second choice, less conspicuous but just as important: the BSD licence. A permissive licence lets a medical-device manufacturer incorporate parts of the library into a closed product, a condition for public funding to yield industrial as well as academic return. The trade-off is that downstream improvements do not necessarily return to the public code — the community takes this as the price of adoption.

Implications

For quantitative clinical research a public, documented, maintained library has structural effects. The entry barrier drops: a small group reaches the same tools as far better-funded laboratories. Reproducibility grows, because a paper can ship together with the executable implementation, and one pipeline becomes the verifiable input of another. ITK already serves as the computation engine for 3D Slicer, the open-source medical imaging application started between Brigham and Women’s Hospital and MIT, which uses ITK for analysis and VTK for visualisation.

The consortium is writing an extended manual — The ITK Software Guide — documenting classes, algorithms and examples, distributed in electronic form. Documentation this complete is unusual for a scientific library at this stage, and it makes ITK usable as teaching material in medical imaging courses.

Limits

Today ITK remains a computation library with no graphical interface of its own: for visualisation you need VTK or an external application. Deformable registration works but costs time and memory, and diffeomorphic schemes are still open research more than stable code. The bindings to scripting languages (Tcl, Python) for rapid prototyping are in progress and immature. The governance model beyond the duration of the NLM contract is not yet formalised in a stable entity: for now the project lives on the public repository (CVS), on the technical mailing lists and on code review among the contractors.

It remains to be seen, over the coming years, whether the public investment will produce the intended effect: shifting the centre of gravity of quantitative medical imaging from closed tools to an inspectable infrastructure, on which research and industry build on the same code.

Cover image: Color cross-sectional anatomical slice through a human head obtained by cryosection, showing internal brain, eyes and skull bone… — photo by National Library of Medicine, Visible Human Project, public domain — https://commons.wikimedia.org/wiki/File:NLM_Visible_Human_Project_-_Section_through_the_head_of_a_human_male.jpg