A desktop workstation built on NVIDIA GB10 Grace Blackwell loads 128 GB of weights into coherent CPU-GPU memory, enough to run quantised models of 70-200 billion parameters without a single byte of clinical data leaving the building. For health workloads the operational ceiling is not capacity: it is memory bandwidth, fixed at 273 GB/s on the module’s LPDDR5x (NVIDIA DGX Spark Hardware Overview).

Context

Health data is a special category under Article 9 of the GDPR: processing is prohibited save for specific legal bases, and sending reports, patient histories or images to an inference endpoint in a non-EU cloud opens a risk surface many providers would rather avoid entirely. As long as LLM inference required data-centre GPUs (80 GB of VRAM per card, four-figure monthly costs, dedicated cooling), the local alternative for a hospital or a diagnostic practice meant a server room. The hardware segment that changes these numbers arrived in early 2026.

NVIDIA GB10 Grace Blackwell puts a 20-core Arm CPU (10 Cortex-X925 plus 10 Cortex-A725) and a Blackwell GPU with 6,144 CUDA cores into a single package, sharing 128 GB of LPDDR5x as unified, coherent system memory (NVIDIA DGX Spark). Nominal performance is 1 petaFLOP at FP4. The chassis measures 150x150x50.5 mm, the power supply is rated 240 W, and the SoC TDP stops at 140 W. Two machines built on this Superchip are on the market: NVIDIA’s DGX Spark and Lenovo’s ThinkStation PGX.

Architecture

For healthcare the decisive point is the memory model. CPU and GPU do not have separate address spaces with a dedicated VRAM to fill over PCIe: they work on the same physical 128 GB pool, with coherence handled in hardware. In practice a 120B-parameter model quantised to 4 bits (~60 GB of weights) fits in memory alongside the KV cache of a long context, without the partitioning that on a discrete 24 GB consumer GPU would make the same model entirely unloadable.

The scheme — Arm CPU, GPU and unified coherent DRAM on a single module, with no hierarchy between system RAM and VRAM — is nothing new in the workstation segment. Apple Silicon brought it to the desktop with M1 on 10 November 2020, and the M2/M3 Max and Ultra configurations with 128-192 GB of unified memory were already viable platforms for local inference of mid-size models, through llama.cpp’s Metal backend or PyTorch’s mps device. On the GB10 the stack changes: CUDA instead of Metal, hence binary compatibility with the higher-class DGX machines and with the library ecosystem grown around CUDA over fifteen years.

For concrete medical workloads this means specialised open models run locally without rewriting anything. BioMistral-7B — Mistral 7B with continued pre-training on PubMed Central Open Access (BioMistral, ACL 2024) — and Meditron-7B/70B — Llama-2 adapted to the medical domain on curated corpora of guidelines and literature (Meditron, EPFL) — are public weights on Hugging Face, runnable with Ollama, vLLM or llama.cpp on the machine’s own CUDA runtime.

The critical point

Faced with 128 GB the first thought goes to capacity: how large a model fits. But for generative inference, perceived latency is governed by a different metric. Each autoregressively generated token forces a re-read of the full set of active weights from memory. The ceiling on throughput in tokens per second is therefore memory bandwidth divided by the size of the weights read per token.

At 273 GB/s with a 60 GB active-weight model, the theoretical ceiling is roughly 4-5 tokens/s before overhead — fine for report summarisation, structured extraction from free text, diagnostic-code classification, but slow to perceive in a conversation. A data-centre H100 exceeds 3 TB/s, about an order of magnitude more. The GB10 favours capacity over bandwidth: in a 140 W desktop the choice is coherent, but it has to be measured against the actual load.

The mitigations live in model selection and the shape of the workload:

  • Smaller, specialised models (a Meditron-7B or BioMistral-7B at ~4 GB quantised) produce tens of tokens/s, enough for interactive assistance, and on many bounded clinical tasks they beat a 120B generalist on cost-effectiveness
  • Mixture-of-Experts models, where only a fraction of the total parameters is active per token, straighten out the ratio between capacity and bandwidth: the model occupies memory, but far less is read per token
  • Batch and asynchronous workloads (overnight reporting, archive indexing, RAG pre-processing) tolerate per-token latency and exploit the capacity without paying the bandwidth

Implications

For a healthcare provider the numbers change. A machine costing roughly 3,000-4,000 euros that sits under a desk, draws as much power as a desktop PC and asks for neither rack nor air conditioning, sustains fine-tuning and inference of open medical models with the data staying inside the perimeter. The legal basis for processing under Article 9 simplifies when there is no transfer to an external processor: no data-processing agreement with a cloud vendor, no adequacy assessment for non-EU transfers, telemetry controllable at the operating-system level.

Binary compatibility with the higher-class DGX machines opens a scaling path without rewriting the stack: a pilot on a single workstation can migrate to wider infrastructure keeping CUDA, the same runtimes and the same weights. For anyone who wants to evaluate on-premise AI before committing budget to infrastructure, it is an entry point at a measurable cost.

Limits

The 273 GB/s is the first constraint to size: a very large generalist fits in memory but generates slowly, and a real-time conversational assistant calls for either a smaller model or an MoE architecture. The nominal petaFLOP is at FP4: the precision actually useful for clinical reporting must be validated per task, because aggressive quantisation degrades non-uniformly and medical tasks carry a low error tolerance.

The GB10 enables inference and light fine-tuning, not training frontier models from scratch — that stays on the higher-class DGX systems. And local hardware does not replace governance: a model running inside the hospital remains a system producing clinical outputs, to be traced, evaluated and kept under policy like any other component of data processing.

Cover image: Oblique view of the NVIDIA DGX Spark desktop workstation: a small gold-colored cube with a perforated metal-mesh finish on a light… — photo by Daniel Lu, CC BY-SA 4.0 — https://commons.wikimedia.org/wiki/File:Nvidia_DGX_Spark_oblique_view_dllu.jpg