A machine learning model trained on radiology images or clinical records does not become clinical software because it reaches good accuracy on a test set: it becomes clinical software when it crosses three separate boundaries — the hospital information systems boundary, the legal boundary on where data may reside, and the regulatory boundary on placing a device on the market. Each one constrains the compute architecture in its own way, and almost always pulls in a different direction from the one you would pick looking at the model’s numbers alone.

Context

In a clinical decision support project the pattern always recurs the same way, easy to describe and laborious to build: a model receives structured data or images from the hospital systems, produces an estimate (a risk score, a segmentation, a classification), and that estimate goes back to a clinician who acts on it or discards it. The model is the part of the whole chain we understand best. The trouble starts at the edges: how data enters, where it is processed, in what form it leaves, with what documentary guarantees.

Integration is dictated by two standards that already exist. For structured clinical data the reference is HL7 FHIR (Fast Healthcare Interoperability Resources), whose Release 4 came out in 2019 with a first portion of content marked normative — including the Patient and Observation resources. FHIR R4 exposes data as REST resources in JSON or XML: anyone who knows web APIs already holds the integration, but the semantic problem stays where it was. An Observation with the wrong LOINC code is syntactically valid and clinically useless. For imaging the reference remains DICOM (Digital Imaging and Communications in Medicine), with DICOMweb exposing the same primitives over HTTP. A computer vision model in production almost always eats DICOM pixels, not PNG files.

Architecture

The decision that weighs most is not which neural network to pick, but where inference runs relative to the data perimeter. Three configurations keep coming back in real projects.

On-premise inference: the model runs inside the hospital infrastructure. Clinical data never leaves the perimeter. The cost is operational — updating models, monitoring performance and managing hardware across dozens of distinct sites becomes a software distribution problem, not a data science one.

Cloud inference, with data crossing the boundary: the model lives on managed infrastructure and clinical data transits through it. The model lifecycle simplifies considerably — a single update point, central monitoring — but the problem shifts from the technical plane to the legal and contractual one.

Distributed training without centralising the data: federated learning carries the model towards the data rather than the reverse. Each site trains locally, and only the parameters (or their updates) are aggregated. Kaissis and colleagues, in Nature Machine Intelligence in June 2020, describe in detail how this family of techniques applies to medical imaging, and what it costs: communication overhead, heterogeneity of local models, the attack surface opened up by malicious updates. The centralisation you avoid on the raw data comes back to you, attenuated, on the gradients.

None of the three is free. The choice is a trade-off between operational cost, data exposure and validation complexity.

The critical point

The constraint that fixes the architecture more than any other is the residency and processing of health data. Under the GDPR, data concerning health is a special category under Article 9: processing prohibited by default, with a narrow set of derogations. This means that “let us move inference to the cloud” is not an engineering decision. It is a decision that requires a legal basis, an impact assessment, and — when the compute provider is a third party — an agreement that qualifies the roles of controller and processor.

On the technical side this means drawing the data flow diagram before the model, not after. Where the data sits in cleartext, where it is encrypted, who holds the keys, in which jurisdiction the compute resides: the answers decide whether the cloud architecture is even tenable. An excellent classifier served by infrastructure that cannot survive a compliance review stays a prototype, not a production classifier.

Implications

On the regulatory front, software that produces estimates used for clinical decisions tends to fall within the definition of a medical device (SaMD, Software as a Medical Device). On 12 January 2021 the US FDA published its first Artificial Intelligence/Machine Learning-Based Software as a Medical Device Action Plan, in response to comments gathered on its 2019 discussion paper. The point that touches architecture is the notion of a predetermined change control plan: an ML-based device that updates over time opens the question of how to authorise a class of model modifications up front without resubmitting every new version. That is an architectural problem before it is a procedural one — it constrains how you version, trace and freeze models.

In practice the three components — FHIR/DICOM integration, data residency, regulatory path — must be designed together. A model trained on data from a single hospital and validated on that distribution rarely survives the move to sites with different equipment, protocols and populations. Cross-site generalisation is the wall most projects run into, and almost never because of the model: it is a data and boundary problem.

Limits

All of this describes constraints, not solutions. The choice between on-premise, cloud and federated depends on factors that change with every project: number of sites, data sensitivity, how often the model is updated, latency constraints, operational budget. Federated learning reduces raw-data exposure but does not zero it out, and introduces coordination complexity that, across few sites, does not pay for itself. The regulatory reference here is EU/US as of early 2021; a real project must verify the framework applicable to its own jurisdiction and to the current date, because at that point both FDA and European practice on AI in healthcare were in full evolution.

On how this overlap between Digital Health, AI and cloud architectures translates into a structured business line, see the noze insight: https://www.noze.it/en/insights/consolidation-ai-cloud/.

Cover image: Server room with aligned rack cabinets and structured cabling, on-premise compute infrastructure for inference — photo by Carl Lender, CC BY 2.0 — https://commons.wikimedia.org/wiki/File:Server_Room_(22397102849).jpg