In 2014 Nick Bostrom published a book I greatly appreciated, Superintelligence: Paths, Dangers, Strategies. Twelve years later the word has entered a US government document, and Elon Musk has announced that he will use it in the name of one of his divisions. In the meantime the meaning has changed, and the useful question is which superintelligence we want to build.

From Bostrom to the SI acronym

Bostrom was already defining superintelligence in 1998 as “an intellect that is much smarter than the best human brains in practically every field”. In the book he distinguishes three forms (speed, collective and quality) and addresses the control problem. Among the possible paths he also examines brain-computer interfaces (BCI), but judges them unlikely: “Brain-computer interfaces look unlikely as a source of superintelligence”. In 2014 Musk recommended the book on Twitter: “Potentially more dangerous than nukes.”

On 29 September 2026 Executive Order 14434 asked federal agencies to use “Super Intelligence” and “SI” instead of “AI”, but with the 2020 definition of AI, which covers everything from recommendation systems to frontier models: far more than Bostrom’s superintelligence. On 4 October Musk announced that SpaceXAI, the division created by SpaceX’s acquisition of xAI, will become SpaceXSI.

What I think

Beyond the words, the direction is towards a superintelligence: AI is already helping to build the AI that comes next, and in Will we all die? I gathered the estimates of extreme risk that come with this race. The question is which superintelligence. In 2014 Musk feared we might be just the biological boot loader of a digital intelligence: the small program that starts the system and then no longer matters. I believe there is another path: super = artificial + human.

It means building superintelligence as a system that people are part of. Agents bring what they do best: speed, scale, memory, the exploration of huge solution spaces, as the 722 mathematics manuscripts produced by an OpenAI model show. People bring intent, judgement about priorities, responsibility towards others and the sense of what is being done. The resulting system is more capable than either part, and that is what makes it super. After all, as I wrote in Intelligence is eating software, our own intelligence is in the end software too.

Bostrom himself provides the foothold. Of the three forms he describes, the collective one lends itself best to including people. His scepticism about BCIs is well founded, and companies such as Neuralink, Musk’s company developing brain-computer interfaces, will have to demonstrate safety and clinical value over time. But the thesis does not depend on an implant. Integration can come through agents working alongside people, through networks of people and systems, through interfaces that today are the screen and the voice and in future may become more direct, as the boundary between code, machines and biology grows thinner, as I wrote about the cell that compiles itself.

Others have pointed in similar directions. In 2015 OpenAI’s mission wanted AI as an extension of people’s will, distributed as widely as possible. In August Mark Zuckerberg, in his letter on personal superintelligence, described it as a tool for each person. These are statements of direction, and they leave two points open: how such a system is built and who answers for it.

Building it with OISG

A superintelligence made of an artificial part and a human part also has a practical advantage: it is easier to govern, because people are already inside the system. In the two-speed world the agentic lane risks pulling away from the human one. In a system like this the two lanes are parts of the same system, and governance is the interface that holds them together: it establishes what the machine decides, what the person decides and who answers for it.

Building a superintelligence like this means deciding, for each property of the system, what belongs to the artificial part and what to the human one. I use the four pillars of OISG, Open, Intelligent, Secure, Governed, which at noze we apply to projects through Admina as well.

PillarArtificial partHuman part
OpenMakes models, training, protocols and logs inspectable and reproducibleVerifies independently and decides what must stay open
IntelligentExecutes approved tasks and choices and explains every answer with sourcesSets the objectives and approves commitments and system changes
SecureResists manipulation, detects and contains incidents while it worksHolds the authority to stop the system and preserve its state
GovernedVerifies compliance automatically and records everything in an immutable logOversees critical decisions and answers for the evidence to authorities

The superintelligence I look at is therefore shared, distributed and governed, because the human part of the system is also the one that has to answer for it.

Cover image: the Turk, the chess-playing automaton built by Wolfgang von Kempelen in 1770, in a 1789 plate by Joseph Friedrich zu Racknitz reconstructing how it worked. Inside the cabinet, among the gears, sat a human player. Humboldt University Library, Berlin — public domain — https://commons.wikimedia.org/wiki/File:Racknitz_-_The_Turk_3.jpg