How accurate a machine learning model is on clinical data is decided less by the architecture and more by how that data was collected, coded, and protected. The work published over the last few months shows it clearly, and it is the first knot to untie for anyone who wants to take a model from the research dataset to the ward.

Context

On 8 May 2018, Rajkomar and colleagues published in npj Digital Medicine a deep learning model trained on 216,221 adult hospitalisations across two academic centres, fed the entire raw electronic health record (EHR), without manual feature selection. The reported metrics — AUROC (Area Under the Receiver Operating Characteristic curve) 0.93–0.94 for in-hospital mortality, 0.85–0.86 for prolonged length of stay, 0.75–0.76 for 30-day unplanned readmission — beat the standard clinical baselines used for the same tasks.

A few weeks earlier, on 11 April 2018, the FDA had permitted marketing of IDx-DR: the first device to apply an algorithm on its own, without review by a specialist, to detect diabetic retinopathy from fundus images in primary care. The clearance rests on a prospective, multi-site pivotal study conducted on those same patients; as I write, the full numeric results of the trial have not yet appeared in a journal.

The two cases have more in common than they seem. In both, the step beyond the earlier academic work lies in the scale and standardisation of the data, not in a genuinely new learning technique.

Data as the dominant constraint

A dataset like MIMIC-III — over forty thousand intensive-care patients from Beth Israel Deaconess, released in de-identified form in 2016 — is valuable precisely because it is rare: almost no clinical data leaves the hospital information system where it is born. Anyone training a model in earnest almost always works on a single centre, with local coding, unaligned vocabularies, and a sampling pattern dictated by the practice of the ward.

Three problems follow that no architectural choice resolves.

Scarcity and imbalance. The outcomes that really matter in the clinic — the complication, the avoidable death, the rare diagnosis — are infrequent by definition. A classifier that always predicts the majority class reaches high nominal accuracy and stays clinically useless. This is why serious work reports AUROC, sensitivity, and specificity separately, and not accuracy alone: these are the metrics that hold under imbalance.

Data not missing at random. In a clinical record, a missing test is not noise: a parameter is measured because someone decided it should be. The pattern of missing values carries the clinician’s judgement inside it. An imputation that treats those gaps as randomly missing erases information and introduces a systematic bias, hard to track down downstream.

Treatment confounding. Observational data records patients who have already been treated. A model that predicts an outcome from variables already touched by the therapies received learns the association between treatment and outcome, not the prognosis underneath. Moved onto an as-yet-untreated patient, it fails systematically.

Validation and transferability

A publishable model and a usable one part ways in how they behave outside the centre where they were trained. The 2018 EHR study reports the metrics of two distinct sites precisely because consistency across hospitals must be demonstrated, not taken for granted. A model tuned to the coding, population, and prescribing habits of one ward can degrade the moment it meets a different distribution — a distribution shift that, in clinical data, is the rule rather than the exception.

The IDx-DR clearance follows the same logic on the regulatory side: the FDA De Novo pathway places the device in class II and requires prospective validation against a pre-specified study, with the endpoints fixed before data is collected. It is the mechanism that stops test-set performance being mistaken for performance on the real patient.

Regulatory constraints on the data

From 25 May 2018 the GDPR applies directly across the Union. For anyone working on health data, two articles weigh more than the rest.

Article 9 places health data among the special categories and prohibits its processing in principle, save on specific legal bases — among them explicit consent, the purposes of preventive medicine or diagnosis, and scientific research with adequate safeguards. The de-identification that makes a dataset like MIMIC shareable is the legal precondition for processing, before it is ever a technical choice.

Article 22 restricts decisions taken solely by automated processing that produce legal or otherwise significant effects on a person. A model that classifies on its own, without human intervention, ends up within this perimeter the moment its output steers a clinical decision. The minimum safeguards — the right to human intervention, to put one’s own point of view, to contest the decision — bind the system’s architecture, not just its documentation: they must be planned upstream, in the way the model enters the clinical workflow.

Limits

The metrics cited come from precise settings — two academic centres for the EHR model — and do not transfer on their own to other populations, coding schemes, or acquisition equipment. None of the work discussed addresses temporal drift: a model trained on historical data degrades as protocols, coding, and population change, and how often it should be revalidated remains an open question. The interpretability of these models also stays partial: AUROC measures aggregate discriminative ability, it does not explain why a single case ended up in a given class — and the explanation of the single case is exactly what the clinician must be able to contest.

How these constraints translate into a continuous practice — predictive analysis, classification models, and clinical decision support — is the subject of the Digital Health line described in noze’s insight: https://www.noze.it/en/insights/ml-health-data/.

Cover image: Fundus photograph of the retina showing scattered dot-blot hemorrhages, signs of early diabetic retinopathy — photo by National Eye Institute, National Institutes of Health, public domain — https://commons.wikimedia.org/wiki/File:Fundus_retinopathy_EDA03.JPG