Almost everything a clinician writes in a record is natural-language text, and making it queryable needs a step that ties those words to stable identifiers in a controlled terminology. MetaMap, which Alan R. Aronson developed at the National Library of Medicine (NLM) from 1994, does exactly this for English: given a biomedical phrase, it returns the UMLS Metathesaurus concepts that appear in it, each with its own identifier.

Context

History, examination, reports, discharge letters: the clinically dense part of a record stays free narrative. The structured portions — demographics, coded diagnoses, laboratory — cover the quantitative side, but the diagnostic reasoning lives in the text. To index that text, search it by concept or aggregate it for epidemiology, the surface terms have to be resolved into entities a machine can work on.

The target of that resolution is the NLM’s Unified Medical Language System (UMLS). Begun in the mid-1980s, UMLS integrates more than a hundred source vocabularies — among them MeSH, SNOMED CT, ICD-9-CM, RxNorm, LOINC — reconciled into a common structure. The Metathesaurus, in the 2009AA release, gathers roughly 2.1 million distinct concepts (UMLS 2009AA, NLM). Each concept carries a Concept Unique Identifier (CUI) of the form C0009443 (common cold), which holds together the synonyms drawn from the different vocabularies. Above the Metathesaurus, the Semantic Network places each concept in one of 135 semantic types — Disease or Syndrome, Pharmacologic Substance, Body Part, Organ, or Organ Component — with relations between the types. The third component, the SPECIALIST Lexicon, is a computational lexicon of English with morphological and syntactic information.

Mapping architecture

MetaMap does not look for exact matches between the text and concept names. It works in stages: it deliberately generates many alternative forms and then scores them. The pipeline, described by Aronson in the founding paper (Proc AMIA Symp 2001), runs like this:

  1. Lexical and syntactic analysis. The text is tokenised and parsed by the SPECIALIST parser, which isolates the candidate noun phrases. Work happens at phrase level, not on isolated words.
  2. Variant generation. For each phrase, MetaMap produces a set of variants: inflections, synonyms from the SPECIALIST Lexicon, acronyms and expansions, derivational forms. This is where heart attack and myocardial infarction end up recognised as the same entity.
  3. Candidate retrieval. The variants are matched against Metathesaurus strings; each candidate phrase accumulates a set of candidate concepts.
  4. Evaluation. Each candidate gets a score that combines four measures — centrality, variation, coverage, cohesiveness — weighing how closely the concept fits the phrase together with the distance between the variants used and the original form.
  5. Mapping construction. Candidates are combined into the best-scoring overall mapping for the phrase; alternative mappings close to the top stay there, as ambiguities.

The output is the annotated text: for each phrase, the assigned CUIs, the score, the semantic type and the position spans. The historical implementation is in SICStus Prolog, distributed as compiled binaries for Linux, Mac OS X and Windows plus source under the UMLS licence, with Java rewrites of the most heavily used components.

The hard part

Where mapping breaks is lexical ambiguity. A short string often rests on several equally plausible CUIs, if you stop at the textual surface alone: cold is a thermal sensation, the common cold, or the acronym for chronic obstructive lung disease. MetaMap’s score is grounded in lexical match and does not separate the three when they tie at the top.

The NLM answered by adding a word sense disambiguation (WSD) module that, on a score tie, picks the semantic type most consistent with the phrase’s domain. The chosen approach — Journal Descriptor Indexing — assigns the text its most likely discipline descriptors and uses them to weight the candidate semantic types; on the ambiguous terms in the NLM test collection, average precision rises from about 0.25 for the baseline to about 0.79 (Humphrey et al., 2006). It remains a statistical, imperfect disambiguation: it helps on the frequent cases, it does not settle the single phrase.

A second problem, independent of ambiguity, is assertion. A concept present in the text may be negated (no evidence of pneumonia), hypothetical, or attributed to a relative. Recognising pneumonia without noticing the preceding no evidence of yields a clinically serious false positive. For this MetaMap integrates negation detection based on NegEx, the regular-expression algorithm of Chapman and colleagues: a list of around 35 negation and pseudo-negation phrases with scope rules. Over 1,235 findings in discharge summaries, NegEx reaches 94.5% specificity and 84.5% positive predictive value (Chapman et al., 2001). It is a deliberately simple method, and its very simplicity limits it on long-range negation.

What it is for

Once made robust on these two fronts, CUI annotation opens uses that raw text shuts out. The original application is indexing: MetaMap is one of the foundations of the NLM’s Indexing Initiative, which assists automatic assignment of MeSH terms to MEDLINE articles. The same mechanism, brought to clinical text, helps code discharge letters and reports towards ICD-9-CM or SNOMED CT, and do algorithmic phenotyping — defining a patient cohort from mentions in unstructured notes.

On top of MetaMap the NLM builds higher-level tools. SemRep starts from MetaMap’s annotation to extract semantic predications — aspirin TREATS pain — from the literature, material for literature-based discovery. The division of labour is clean: MetaMap identifies the entities, the upper layers structure their relations.

Limits

The firmest constraint is the English orientation. The SPECIALIST Lexicon and variant generation rest on English; the Metathesaurus holds terms in other languages, but the pipeline was not built to analyse Italian text directly. The route taken in research translates the text by machine first and then passes it to MetaMap, with accuracy lost at each step. Three operational constraints remain:

  • Uneven coverage: extraction is more accurate where source vocabularies are mature and dense; specialist niches fare worse.
  • Computational cost: the Prolog pipeline is slow on large volumes; running it in batch over millions of documents needs parallelisation and pre-filtering.
  • Licence: use requires the UMLS Metathesaurus License Agreement, free but with registration, and with specific obligations inherited from sub-vocabularies (SNOMED CT, CPT). Outside the academic setting, it is worth checking first.

For anyone working on English biomedical text today, MetaMap and UMLS are the infrastructure one measures against; for clinical Italian the space stays open, and translating the text before mapping it is a stopgap that adds error across two steps rather than removing it.

https://metamap.nlm.nih.gov/ https://pubmed.ncbi.nlm.nih.gov/11825149/ https://www.nlm.nih.gov/research/umls/ https://www.nlm.nih.gov/research/umls/archive/archive_home.html https://pubmed.ncbi.nlm.nih.gov/12123149/ https://pubmed.ncbi.nlm.nih.gov/19890434/ https://www.noze.it/en/insights/metamap-umls-nlp-biomedico/

Cover image: Modern multi-storey brick-and-glass building with an entrance staircase, home of the National Library of Medicine on the NIH campus… — photo by Tony Webster, CC BY 2.0 — https://commons.wikimedia.org/wiki/File:Lister_Hill_Center_-_Building_38A_-_National_Library_of_Medicine_at_NIH.jpg