On 10 August 2026 Vass Bednar published Google Search Is Dying. What Comes Next Is Worse in The Walrus, updated on the 11th. The deck carries the thesis: “As AI eats the web, the internet’s collective memory is disappearing”. The article lines up the decay of results, the traffic Wikipedia loses because AI systems reuse its content without sending anyone to the site, the Internet Archive’s difficulties and the deletion of editorial archives; and it proposes that Canada treat search as infrastructure, “rather than a ‘free’ consumer service”.

The article presents the thesis without data on the scale of the search crisis. It proposes treating search as public infrastructure at the level of the default engine; the index level, where infrastructure decisions actually matter, remains secondary in the analysis. Below are the figures of the crisis and the state of Europe’s open index.

The figures

Link rot. The Pew Research Center examined about one million webpages, ninety thousand for each year from 2013 to 2023: 38% of the pages that existed in 2013 are no longer reachable, as is 25% of the whole sample. Among pages on news sites, 23% contain at least one broken link; among government pages, 21%. Across 50,000 English Wikipedia entries, 54% carry at least one reference pointing to a page that no longer exists, and 11% of all references are unreachable. The sample comes from a Common Crawl snapshot of March–April 2023, a detail that matters later.

Wikipedia. On 17 October 2025 the Wikimedia Foundation published the figure after correcting its bot detection: human pageviews for May and June 2025 fell by “roughly 8% as compared to the same months in 2024”. Marshall Miller ties it to the fact that “search engines are increasingly using generative AI to provide answers directly to searchers rather than linking to sites like ours”.

FiveThirtyEight. The site closed in March 2025 with the newsroom laid off. In mid-May 2026 the archive vanished, and on 19 May Nate Silver did the arithmetic: “about 200,000 person-hours of work that ABC News just deleted”. Checking costs one line of curl: on 13 August 2026 fivethirtyeight.com, the project addresses and the individual article addresses all answer 200 after a redirect to abcnews.com/politics. In place of each piece stands the politics section home page, so not even a 404 to tell a crawler the page is gone.

The default engine

On 4 June 2026 the European Parliament changed the default engine in its institutional browsers, Edge and Firefox, from Google to Qwant: 720 members plus administrative staff, with Google still selectable by hand. This is the move Bednar proposes for Canada.

The liability question moved in parallel. On 28 May 2026 the Landgericht München I, case 26 O 869/26, held that AI Overview answers are Google’s own content — “an independent presentation for which [Google] is responsible” — and ruled out the exemptions of Articles 4–6 of the Digital Services Act, because the system “independently compiles the information” rather than merely displaying material from others. The decision is not final.

Who holds the index

The default engine determines who phrases the answer. The index the answer is built on sits one layer below, and substitution there is slower: Qwant runs its own crawler, strong on French and European content, and for global coverage it long supplemented results with Bing’s.

Since 2025 that dependency has been shrinking commercially. Qwant and Ecosia set up European Search Perspective and built Staan, an entirely European index which since August 2025 serves roughly 30% of queries in France and since July 2026 operates in Germany too, with API access open to other engines and to AI applications.

The open artefacts

At the index layer there are two open artefacts, and they do different things.

Common Crawl is a corpus. A 501(c)(3) foundation started in 2007 publishes over 300 billion pages spanning fifteen years, with 3–5 billion new pages a month, freely downloadable. It is the raw material others build on, and it is the base Pew measured link rot from: without that corpus the measurement would not exist, nor would the discussion that follows it.

Open Web Index is an index. It comes out of the OpenWebSearch.eu project, Horizon Europe programme, grant agreement 101070014, coordinated by the University of Passau with eleven participants and funded with €8,502,621.75. It offers roughly one petabyte of web data, with programmatic access at openwebindex.eu under a licence reserved for research and development projects.

On CORDIS the project is listed as closed since 28 February 2026. The index remains in a test phase and continues to be updated, and the project’s status page publishes no continuation plan. European administrations are changing their default engine in the same half-year in which the continent’s only open index has run out of the agreement that paid for it.

A proposal

On openness I hold an old position and not a neutral one, and I will keep to it here: of the two routes the second is the one that holds, and not because it is cheaper. A European commercial index solves the geographic dependency and leaves the structural one standing, because whoever pays to query someone else’s index cannot check what it leaves out. An open index can be queried, counted and contested, which is the same property by which an artefact counts only if somebody can verify it.

Four things are already available and call for a decision rather than an invention.

  • Refund the open index. The infrastructure exists, runs and produces a petabyte; the agreement expired in February after three and a half years and €8.5 million. Putting it back on a budget is an administrative decision, and the technical cost of restarting from a standstill would exceed the cost of not stopping.
  • Widen the licence. Access restricted to research and development keeps out precisely the people who would build the engines meant to use it. The useful precedent is Common Crawl, open to commercial use as well, which is why people have been building on it for fifteen years.
  • Put it where preservation lives. National libraries already hold the mandate and the budget for legal deposit, digital part included. A public index of the web belongs inside that perimeter, next to those who archive, rather than in a fixed-term consortium.
  • Procure with the index in view. An administration changing its default engine can ask in the tender which index answers, and on what terms. It is the same question that elsewhere separates a policy from an architecture, and it changes the incentives of whoever is selling.

Meanwhile the part that depends on each of us is small and concrete: archive what you cite. The FiveThirtyEight pages that survive are the ones somebody had sent to the Wayback Machine before the redirect.

Limits

The Pew figures measure 2013–2023 and describe the web of that period. Wikipedia’s 8% is a comparison between two months following a revision of bot detection, so it indicates a direction rather than a time series; the Foundation presents it that way itself. On the Munich ruling I have read the decision as reported and not the case file, and it is not final. I have run no queries against the Open Web Index, so on that index’s quality and coverage I say nothing: I verified the project’s administrative status, the declared licence and the announced volume. If continuation funding was decided after February and is not published where I looked, this part of the article ages well and gladly.

Cover image: card catalogue of the Biblioteca Roncioniana in Prato, photo by Sailko, 2020 — CC BY-SA 4.0 — https://commons.wikimedia.org/wiki/File:Prato,_biblioteca_roncioniana,_scalone,_schedario_02.jpg