The Predictions Ledger

Fifty-six dated bets, how to score them, and why most of them cannot show that the author knows anything. Nobody has agreed to do the scoring.

2 November 2026

The first bet in this book falls due on 15/03/2027: a certificate rule already on the calendar will take effect when the calendar says. I expect it to be confirmed, and that will teach nobody anything about me. A ledger that counts that row as a win is worse than no ledger, because it turns inertia into a track record.

Thesis. Score this book against a forecaster who says that nothing will change. On a no-change bet the book and that forecaster say the same thing, so the bet adds nothing to the book's score, right or wrong. Only 27 of the 28 change and mixed rows can show skill, the exception being REVOKE-3, and 13 of those can end with no result, ten of them if nobody runs the test. I expect the book to lose this comparison. The file predictions.json with its checker holds the bets still. Nobody has committed to score them.

The most contestable claim is the first: that a correct no-change bet earns nothing. The first objection below is the case against it.

A disclosure. I build AI systems for law firms and I sell the discipline this book recommends. A clean ledger would be useful to me, so check the scoring rule yourself.

What is in the file

predictions.json, at the root of the book's repository, holds 56 entries: 43 from the 15 new chapters counted in Where This Book Is Wrong, 4 from that chapter, 3 from this one and 6 from the two Prologue essays. Floating Theories also has dated predictions, and Who Gets to Write the Specification?, Many Hands Make Heavy Work and Systemic Coping Syndrome state predictions, mostly with horizons counted from the start of an experiment. Nobody Is Paid to Delete predicts what a measurement will find, with no date. I left those five out, so the ledger covers 19 chapters and not the whole book.

Each entry has an id, a chapter, a claim, a resolveBy date, a refutedIf condition and a status, open for all 56. Claims and conditions are copied word for word. Three fields are my reading: class (no-change, change, mixed or meta), shape (the kind of bet) and voidable (whether it can end with no result).

The counter-chapter counted 23 no-change, 16 change and 4 mixed. The file gives the same split, and nine of the 43 can end void or untested. Reaching 16 took two judgement calls. REVOKE-3 says something will happen, but only that a published schedule holds. PAYS-3 says any future encoding "will be free to read", and it ends untested if no such encoding exists. By what they risk, the first belongs with the no-change bets and the second is close to no bet.

Across all 56 the split is 25 no-change, 22 change, 6 mixed and 3 meta, the meta rows being bets on how other rows score. Thirteen state a way to end void or untested. Five more need someone to run a study or a scoring and do not say what happens if nobody does, and I mark those "implied".

Which bets are cheap

A bet is cheap when the author can collect without the world doing anything he foresaw. There are three kinds here.

  1. Wins by default. All 25 no-change rows, and REVOKE-3.
  2. Wins on one hit. Eight of the 16 change bets in the counter-chapter's count say that at least one report, paper or decision will exist somewhere. These at least die if nothing happens, so they beat the first kind.
  3. Has an exit. The 18 void-able rows. Of the 12 change bets that name a measurement, 11 can end with no result. AGENT-2 alone says that no measurement by the date refutes it. The bets most exposed to a number are the ones with a door at the back.

So, by my reading, the most exposed rows are few: AGENT-1, AGENT-2, TRANS-2, APPEAL-1, MATHS-3 and WRONG-1. Each needs something specific to happen and has no exit if it does not.

The base rate for no change

The direction is supported and the size is not. Three figures, none measuring what I need:

None of these samples the things an essayist thinks an institution ought to have done, and my horizons, up to just over four years, should favour change more. I found no base rate for that population. The scoring rule leans only on the ordinal claim that no-change bets start above evens, not on any of the three numbers.

That is enough to deflate my own second prediction in the counter-chapter (WRONG-2). Its first limb says at least 80% of the 23 no-change bets will stand confirmed. Eighty per cent sits inside the range above, so it shows skill only if my rows are far more change-prone than those samples, and I have no measurement that they are. Two of the 23, GOOD-1 and APPEAL-3, are not due until 2029, so if "stand confirmed" needs a closed window, only 21 can count and 19 are needed. Its second limb says fewer than half of the 16 change bets will be confirmed, and counts voids as unconfirmed. PAYS-3 is not due until 2030 and five more can void, so that limb starts six rows ahead.

How to score it

  1. Resolve each row by its own words. When its date passes, mark it confirmed if every limb of the claim holds, and otherwise refuted, void or untested using the stated condition, and record the evidence. A claim that fails in a way its condition does not name is refuted. A row marked "implied" is void if the test it needs was not run. WRONG-4 has two readings; score it as the share of all wrong answers in the band against the share of all right answers. A row that turns on finding nothing, whether that refutes it or confirms it, means a search, so a confirmed absence is only ever a failure to find. Four rows of that kind name where to look (LIVE-3, AGENT-3, REVOKE-2, SUMM-2); the rest do not, and the scorer should state the search used.
  2. Never publish a total. Report no-change, change, mixed and meta apart, and give meta rows no credit: WRONG-2 and LEDGER-3 collect when the change bets lose, so counting them would pay me both ways.
  3. Compute skill against a no-change forecaster. Add one for each change or mixed row confirmed, and take away one for each refuted. No-change rows add nothing, because that forecaster said the same, and so does REVOKE-3, because that forecaster also says a published schedule holds. On the 11 measured change rows with an exit, "nothing will change" does not say what the number will be, so that forecaster has no view; report those points apart, because they record a correct call and not a win over it. This is a reference forecast, which is the idea behind a skill score in weather forecasting (Murphy, 1988; I confirmed the record, not the text), reduced to counting because I gave no probabilities. A mixed row counts as a change bet. The script's --score flag computes this.
  4. Report the void rate beside the score. A void is not a miss, but a high rate means the author's riskiest claims were never tested.

This makes the case against a book of pure no-change bets exact. Its skill against this forecaster is zero by definition, whatever happens. It cannot lose and it cannot win, and it says nothing about what will move. Daniel Kahneman, in the same class, describes a forecaster who gives everything that happens .6 and everything that does not .4: "your calibration is perfect but your discrimination is miserable". A high hit rate on no-change bets is at best the first half of that sentence.

My expectation is below par. LEDGER-3 bets that refuted rows will outnumber confirmed ones among those due by 31/12/2028, for 27 rows, 12 of them with exits. If I am right, the book forecasts change worse than saying nothing will change.

How the ledger is kept, and what nobody has promised

The mechanism is a file and a script, scripts/check_predictions.py. It exits non-zero if an entry lacks a refutation condition of at least four words, has a resolveBy on or before 02/11/2026, or carries a date that does not appear in its own text. It does the same if a row's sentences and those of its numbered item in the chapter are not the same set, if the counts printed here or in the counter-chapter disagree with the file, if a resolved entry has no evidence, or if the table below differs from the file. build.sh runs it, so the book does not build with a bad ledger. It lists rows past their date and still open, and it does not fail on them. The output below is from a run on 02/11/2026, copied.

$ scripts/check_predictions.py --self-test
self-test: 18 bad cases, 0 not caught
$ scripts/check_predictions.py
ok: 56 entries, all with a refutation condition and a date after 2026-11-02

The check is a script because a Lean encoding would only restate the JSON schema. It does not decide whether a refutation condition rules anything out: four words of nonsense, printed in the chapter and copied here, would pass. The table carries a SHA-256 of the fields that must not move, so a later copy of the file can be compared with the printed book.

Nobody has committed to review this ledger on any date. I have not, no institution has, and no timer does it. The overdue list prints only when someone runs the script or pushes to the repository. Three rows (PROOF-1, TRACE-3, APPEAL-2) say "I will say so" if they go untested, and nothing enforces those sentences. On the print date the repository is private, so you cannot read the file. The printed table is the copy that counts, and anyone can score from it. LEDGER-2 bets that no one without a stake in the book will.

What exists and what is proposed

Exists: the file, the script, its 18 tests with bad input, the wiring in build.sh and the generated table. Does not exist: a resolved entry, any scoring, a stated probability on any row, a search protocol for rows resolved by absence, a consensus forecast to compare with, or a scorer. Proposed: scoring by kind, skill against a no-change forecaster, and a credence field filled in before any row resolves. Proved in Lean: nothing.

Long Bets, started in 2003, describes itself as "an arena for competitive, accountable predictions" (Long Bets), and holds stakes (Long Bets FAQ), which this ledger lacks. The tournaments did something harder: "IARPA (the sponsor of the forecasting tournament) chose the forecasting questions" (Satopää, Salikhov, Tetlock and Mellers, manuscript). Here the author chose them, the largest defect no checker reaches. What this adds is small: bets bound to the text of an argument, with drift made a build failure.

Goodhart. Optimise for this ledger and you write more no-change rows, widen each at-least-one bet, and add void clauses. Scoring by kind removes the first. Nothing mechanical removes the others, so the defence is printing each bet's kind for a reader to discount.

What formalising loses. A row flattens a paragraph. AGENT-1 is two bets with two dates under one resolveBy, so its first date goes in a checkpoints field. Some baselines and consequences move to a note that the table omits.

Objections

"A no-change bet against a crowd that expects change is the bravest bet there is." Correct. Information depends on the prior, not the polarity, and a no-change forecaster is a floor, not a consensus. Verdict: accepted, and my thesis holds only against that floor. A posted market or tournament question on these rows would change the scoring: that price becomes the reference, and no-change rows priced below evens earn credit.

"Bare claims have no probabilities, so no proper score applies." Correct. Proper scoring rules "encourage the forecaster to make careful assessments and to be honest" (Gneiting and Raftery, 2007), and they need a probability to work on. Verdict: conceded. The chapters state none, and counting by kind is the most these rows support. Credences stated before the first row resolves would replace it.

"The author picks the questions, classifies them, and may resolve them." Yes, and two of the classes that make the counter-chapter's numbers work are arguable. Verdict: holds, and no script fixes it. The digest stops rewording, not bad choices. I would treat the classes as worthless if an independent scorer moved more than a quarter of the rows.

"The book's argument is not a forecast, so this is theatre." The counter-chapter already says the book fails as a forecast of what gets bought. Verdict: partly. The ledger does not test the book's distinctions. It tests whether the author can see where institutions will move, and an author selling a view about institutions should be held to that. I would call it theatre if rows with stated costs were refuted and no chapter changed.

Predictions

  1. By 30/06/2029, any published scoring of this ledger that covers every entry due by 31/12/2028 will show at least half of its void-able entries due by then ended void or untested. Refuted by a published figure below half. Void if no such scoring is published.
  2. By 31/12/2028, nobody with no stake in the book will have published a scoring of ten or more entries of this ledger. Refuted by one such scoring from anyone who meets that description.
  3. By 30/06/2029, in any published scoring of this ledger that covers every entry due by 31/12/2028, refuted entries will outnumber confirmed ones among the change and mixed entries due by then and not void. Refuted by confirmed entries equalling or outnumbering refuted ones. Void if no such scoring is published.

The second is cheap in exactly the way this chapter complains about. If it holds, everything above is an instruction nobody followed.

The ledger at print

Generated from predictions.json by scripts/check_predictions.py --write-table. SHA-256 of its frozen fields at print, in groups of 16: c39bece590849ee6 e6410a12ad7df626 bbf19804de48772c f1e9c9f32d247c4e.

Key to the prefixes: MATHS, Mathematics Went First; CHECK, The Checking Threshold; PROOF, The Proof Passed. The Wrong Thing Ran.; LIVE, Never Breaks a Rule, Never Finishes; TRANS, The Translator Writes the Rule; DISC, Disclosure Without a Predicate; GOOD, Perfect Compliance, Defeated Purpose; HANDS, The Hands; QUERY, Every Query Was Allowed; AGENT, Who Proves the Agent Is Yours?; REVOKE, Yesterday's Permission; SUMM, A Summary Is Not a Permission; TRACE, A Trace Is Not a Replay; APPEAL, A Proof Is Not an Appeal; LAST, The Last Person Who Knows How; PAYS, Who Pays to Keep It True?; WRONG, Where This Book Is Wrong; LEDGER, The Predictions Ledger; AGENDA, The Research Agenda.

ID By Kind Claim and how it is refuted
MATHS-1 31/12/2027 change, void-able (implied) Library statements drift less. Take every AI-produced Lean formalisation of a top-line result released by an AI developer from 6 October 2026 to the end of 2027, starting with OpenAI's repository. Classify each by whether its main statement uses only definitions already in Mathlib at the time of release, or introduces new ones. Within each subject area, pair statements across the two classes by the length of the main theorem statement, not counting the definitions it introduces, and draw 50 pairs at random (or all pairs, if there are fewer); two people who read Lean and did not write the statements compare each with the paper's headline claim, counting statements that are weaker, stronger or different, with a third reader settling disagreements and agreement between readers reported. The mismatch rate in the new-definitions class will be at least twice the Mathlib-only rate and at least five percentage points higher. Test. If the rates are similar, the library is not doing the anchoring I claim, and the ordering argument loses its mechanism.
MATHS-2 31/12/2027 no change Canonisation lags generation. By the end of 2027, no complete proof of Fermat's Last Theorem whose non-Mathlib part was mainly written by AI will compile with under 1.3 million non-blank lines that were not already in Mathlib on 4 September 2026, about a tenth of the thirteen million lines Anthropic reports (Anthropic does not say how it counted; the bar is the fixed 1.3 million, whatever a recount of Anthropic's proof shows). Test. If one does, AI canonisation has caught up with AI generation faster than I expect, and Kontorovich's gap, with the Nobody Is Paid to Delete reading of it, has closed.
MATHS-3 31/12/2028 mixed In law, official statements come before accepted proofs. Before the end of 2028, at least one tax or benefits authority that publishes its rules in executable form will, for some filing or eligibility decision, rely on an artefact submitted by the applicant or their software that a checker run by the authority verifies against the authority's own published executable rules, in place of recomputing the decision itself, as shown by its published guidance or legislation (checking only a filing's format against a schema does not count), and no authority will accept one against a vendor-written statement of its rules that it has not adopted, meaning published as its own statement of those rules or certified after its own review. Test. If the first half fails, a statement library is not enough to start the transition in law. If the second fails, authority over the statement matters less than I claim.
CHECK-1 31/12/2027 change, void-able (implied) Independent checkers beat more copies, at the handoffs. On long sequential tasks where a machine checker exists for every task, so that the task set is held fixed, compare four arms at the same total compute, counting the checkers' compute inside the budget: identical copies checking each other; copies with varied prompts and roles checking each other; copies whose handoffs are checked by a different model family within a few points of the producer on a named benchmark, so that independence is not confounded with competence; and copies whose handoffs are checked by a machine checker the agents cannot edit. Measure output per agent, defined as tasks completed per agent-hour at a fixed acceptance rate, as teams grow from 1 to at least 64 members, and catch rate on injected honest errors that the producing model rates as plausible. The two independent-checker arms will show higher output per agent at large sizes than the identical-copy arm, and the gap will widen with size; the prompt-diverse arm will sit between them. Horizon: published results by the end of 2027. Test. If identical copies match the independent-checker arms at large sizes, I am wrong.
CHECK-2 31/12/2028 change, void-able (implied) Machine-checked handoffs turn the curve before domain depth does. Within a single domain with real test suites, such as software maintenance on public repositories, teams whose handoffs are machine-checked against properties the agents cannot edit will be the first to show output per agent rising with team size beyond 32 agents, in a published and independently replicated result, before teams of the same model and size without such checks in the same domain. Across domains, the first such result in a domain without machine checks will come later than the first in a domain with them. Horizon: the end of 2028. Test. A replicated case of convex scaling without handoff checks arriving first would refute this.
CHECK-3 31/12/2028 change, void-able (implied) Verification failures fall more slowly than coordination failures. Annotate failures in successive model generations with the MAST taxonomy, with the same frozen annotator throughout, using the MAST version 3 breakdown as the baseline. Verification failures per task will fall more slowly across generations than inter-agent misalignment failures per task. Horizon: two model generations, or the end of 2028. Test. If verification failures per task fall as fast as misalignment failures, the checking term is not the binding one.
PROOF-1 31/12/2028 change, void-able By 31/12/2028, the first published study of defects across at least three formally verified AI-agent or LLM tool-wrapping systems will find most defects in specifications, adapters or tool descriptions, not in proved policy code. Test. Refuted by such a study finding the majority inside proved policy code. If no such study exists by then, the prediction is untested and I will say so.
PROOF-2 31/12/2027 no change On 31/12/2027, the current Model Context Protocol specification will still not require verifiable declarations of a tool's effects. Test. Refuted by a published revision making them normative for clients or servers.
PROOF-3 31/12/2028 no change, void-able By 31/12/2028, none of Amazon, Google, Microsoft, OpenAI or Anthropic will publish, for a production agent guardrail described as formally verified, a list of what lies between the proof and the deployed code in the manner of seL4's assumptions page. Test. Refuted by one such publication. Untested if none of them describes a guardrail that way.
LIVE-1 31/12/2027 no change By 31/12/2027, neither LangGraph's nor the OpenAI Agents SDK's documentation will describe a built-in termination condition that takes a calendar date and a human lead time and hands over automatically. Test. Refuted by either documenting one.
LIVE-2 31/12/2027 change, void-able By 31/12/2027, any published benchmark that adds a dated obligation to blocked agent tasks will find that, even with an escalation tool, more than 5% of blocked runs fail to hand over by the last useful tick. Test. Refuted by such a benchmark finding 5% or fewer. Void if none is published.
LIVE-3 30/06/2028 change By 30/06/2028, a judgment, a regulator's decision or an insurer's published claims review in England and Wales or Australia will attribute a missed legal deadline to an automated agent or tool that stopped without telling anyone. Test. Refuted if a search at that date of BAILII, AustLII, SRA decisions and the main insurers' reviews finds none, which would suggest that the diary is catching this, or that nobody is recording it.
TRANS-1 31/12/2028 no change By 31/12/2028, no peer-reviewed study will show a language-model translator, not prompted about ambiguity, flagging at least half of the interpretive choices that a lawyer and programmer pair flags on the same statutory provisions. Test. A study reporting that result refutes this.
TRANS-2 31/12/2028 mixed By 31/12/2028, CNAF or the French tax authority will have stated publicly that a named production calculation runs on Catala, and no national tax or benefits agency will have stated that a production calculation has its rules encoded in a general-purpose proof assistant such as Lean, Rocq or Isabelle. Test. Either a missing Catala statement or such a proof-assistant statement refutes this.
DISC-1 31/12/2027 no change By 31/12/2027, no OAIC-published enforcement action under APP 1.7 will rest on the published policy alone. Each will cite information from the entity, a complaint or other outside evidence. Test. Refuted by one such action whose stated evidence is only the policy text, which would mean a document-only test is more useful than no_test_on_policy_alone suggests.
DISC-2 30/06/2027 no change On 30/06/2027, in the 30 alphabetically first ASX 50 companies that publish an APP 1.7 section (or all that do, if fewer, with the threshold at a majority), fewer than 16 will state, for each kind of decision, whether a program makes it solely. Test. Refuted if that threshold is met, which would show my reading of the model text overstates how the market will follow it.
DISC-3 10/12/2027 no change By 10/12/2027, neither the OAIC nor an Australian court will have said in terms whether APP 1.8(b) and (c) require separately identifiable lists. Test. Refuted by any such statement.
GOOD-1 02/12/2029 no change By 02/12/2029, no EU market surveillance authority will have published a decision that a high-risk AI system in conformity with a cited harmonised standard nonetheless failed the requirement that standard covers. Test. Refuted by one such decision, which would show rebuttal working as the essay said it would.
GOOD-2 31/12/2028 no change By 31/12/2028, no harmonised standard cited in the Official Journal under the AI Act will require either a test instance withheld from the provider or field sampling by someone the provider does not select. Test. Refuted by one such requirement, which would show the drafters ahead of me.
GOOD-3 31/12/2028 change By 31/12/2028, a published decision of a regulator or court in the EU, UK or US will treat an AI system's behaving differently under evaluation than in use as a ground for action. Test. Refuted if none does, which would suggest evaluation sensitivity matters less in practice than this chapter claims.
HANDS-1 31/12/2027 no change By 31/12/2027, no agent runtime with at least 10,000 GitHub stars will have published a machine-checked theorem stating confinement for a whole session, covering the composition of its checks, about the code it ships. Individual-check theorems, like those in L3M, do not count. Test. Refuted by a public theorem of that form, with a proof that rebuilds.
HANDS-2 31/12/2028 change By 31/12/2028, a public incident report or peer-reviewed paper will document information passing, in a system that separates control from data, through values accepted by a schema or an allowlist on a quarantined component's output, with the channel carrying data the system's policy meant to withhold. A contrived laboratory protocol in which sender and receiver are both instructed to signal does not count. Test. Refuted if no such report exists by then.
HANDS-3 31/12/2027 change By 31/12/2027, a control-and-data-separating design will report benign utility within three percentage points of its undefended baseline, overall across all four AgentDojo suites, using the same model, with enforcement on and a policy denial counted as a failed task, and with no prompt injection succeeding inside its stated threat model. Test. Refuted if none has by that date.
QUERY-1 31/12/2027 no change By 31/12/2027, no IETF RFC or working-group draft on AI agent authorisation will define a normative check over the cumulative answers a principal or delegation chain has received. Test. Refuted by one such text, adopted by a working group, that requires it.
QUERY-2 30/06/2028 change By 30/06/2028, a peer-reviewed paper or vendor-acknowledged report will show an LLM agent inferring a restricted fact in an enterprise retrieval system purely through queries its per-document permissions allowed. Test. Refuted if no such demonstration exists by then.
QUERY-3 31/12/2028 no change By 31/12/2028, none of Microsoft 365 Copilot, Google Workspace's Gemini or Glean will document a default per-tenant budget or pooled disclosure ledger for answers over tenant content. Test. Refuted by that documentation from any one of them.
AGENT-1 31/12/2027 mixed By 30/06/2027, the IETF OAuth or WIMSE working group will have adopted a draft that requires each hop of an offline-verifiable agent chain to narrow, and by 31/12/2027 no such document will have been published as an RFC. The delegate SD-JWT draft as it stands does not count, because limits there are optional. Test. Refuted if no such adoption occurs by the first date, or if an RFC appears by the second.
AGENT-2 31/12/2027 change By 31/12/2027, a published measurement of deployed agent credentials, stating its sample and method, will find that most root or initial grants allow at least twice the actions the tasks observed used. Test. Refuted if such a measurement finds otherwise, or if none has been published, in which case my claim that this matters enough to be measured has failed too.
AGENT-3 30/06/2028 change By 30/06/2028, a public incident report, advisory or regulator decision will describe harm in production, not a researcher's demonstration, in which one AI agent obtained an effect through a second agent's own valid credential that the first agent's credential did not allow. Test. Refuted if a search at that date of the AI Incident Database, CVE records, and ICO and EDPB decisions finds none, which would suggest cross-agent deputies matter less than I claim, or that nobody is reporting them.
REVOKE-1 31/12/2027 no change By 31/12/2027, no RFC or active Internet-Draft from the IETF WIMSE working group will require invalidation of artefacts derived by an agent under a revoked authorisation, as opposed to tokens, sessions and cached decisions. Test. Refuted by any such text with a normative requirement on derived outputs.
REVOKE-2 30/06/2028 change By 30/06/2028, a public incident report, regulator decision or judgment will attribute harm to an AI agent acting on authority that had already been withdrawn or revoked at the time it acted. Test. Refuted if a search at that date of ICO and EDPB decisions, BAILII and the AI Incident Database finds none, which would suggest either that the window matters less than I claim or that nobody is measuring it.
REVOKE-3 15/03/2027 change The 100-day certificate step in the Baseline Requirements will take effect on 15/03/2027 as scheduled. Test. Refuted if the CA/Browser Forum delays or withdraws it, which would weaken this chapter's account of the web shifting weight from revocation to shorter lifetimes.
SUMM-1 31/12/2027 no change By 31/12/2027, no published run on EAL-Bench or a declared successor will show incremental typed memory, with no rebuild, and with neither the ledger nor the full history available to writer or gate, reaching at most 2% unauthorised actions and at least 95% legitimate actions in all three domains. Test. Refuted by one such result, which would also refute my third claim.
SUMM-2 30/06/2028 change By 30/06/2028, a public incident report, regulator decision or judgment will attribute an unauthorised act by an agent in an organisation to a permission or restriction altered by the agent's own memory or context compaction, with no attacker involved, in an incident occurring after this chapter's date. Test. Refuted if a search at that date of ICO and EDPB decisions, BAILII and the AI Incident Database finds none.
SUMM-3 31/12/2027 no change By 31/12/2027, no ICO guidance, including the agentic AI guidance its plan lists as in drafting, will state whether erasing a source reaches a summary derived from it in an agent's memory. Test. Refuted by any ICO guidance text giving either answer.
TRACE-1 31/12/2027 no change By 31/12/2027, none of OpenAI, Anthropic or Google will publicly guarantee byte-identical outputs for repeated identical requests to a hosted model. Test. Refuted by a public documentation or contract term making that guarantee for a named model.
TRACE-2 31/12/2027 no change On 31/12/2027, VLLM_BATCH_INVARIANT will still default to off in vLLM's main branch. Test. Refuted by the default being on, or by batch invariance becoming unconditional.
TRACE-3 31/12/2028 no change, void-able By 31/12/2028, no published ICO statutory code on AI and automated decision-making, and no final EU harmonised standard on AI logging, will require deployers to be able to regenerate a model's output. Both will ask for records. Test. Refuted by such a requirement in the published text of either. If neither is published by then, the prediction is untested and I will say so.
APPEAL-1 31/12/2027 mixed By 31/12/2027, the Criminal Procedure Rules for England and Wales will contain an express provision on computer-generated evidence, and no statute will have abolished the common-law presumption in criminal proceedings. Test. Refuted by a statute abolishing it, or by no such rule being in force on that date.
APPEAL-2 31/12/2028 mixed, void-able The Law Commission's final product liability report, if published by 31/12/2028, will retain a rebuttable presumption of defect in some form and will give a defendant no route to displace it by tendering a specified form of machine-checkable evidence. Test. Refuted by the published text. If no report appears by then, the prediction is untested and I will say so.
APPEAL-3 31/12/2029 no change By 31/12/2029, no reported judgment of the High Court or above in England and Wales will rest a finding about what an AI agent did on a machine-checked proof certificate. Test. Refuted by one such judgment.
LAST-1 31/12/2028 change, void-able By 31/12/2028, a published study of lawyers, with a baseline and repeated unaided tests spanning at least 12 months under generative AI, will find juniors with access scoring below controls on unaided judgement. Test. Refuted by such a study finding no difference or a gain. Void if none is published.
LAST-2 31/12/2028 change, void-able By 31/12/2028, the first published test of lawyers reviewing AI legal drafts with planted errors at 5% prevalence or below will report mean detection under 80%. Test. Refuted by a mean of 80% or above. Void if none is published.
LAST-3 31/12/2028 no change By 31/12/2028, neither the SRA nor the Bar Standards Board will require regulated firms or chambers to test detection of errors in AI output with known errors. Test. Refuted by either doing so, in a rule or in guidance saying firms should.
PAYS-1 31/12/2028 no change By 31/12/2028, no formal library of statute law in any jurisdiction will publish a recurring maintenance budget covering more than one legal domain, funded by anything other than grants, one employer or one government department's project money. Test. Refuted by one such library with a published budget and named editors.
PAYS-2 31/12/2027 change, void-able Mathlib's open pull requests on GitHub will number more at 31/12/2027 than at 01/10/2026, though more are closed in 2027 than in 2026. Test. Refuted if they number fewer, which would weaken my claim that cheap generation raises arrivals faster than review capacity. Untested if closures fall.
PAYS-3 31/12/2030 change, void-able Any formal encoding of a rule given a presumption of conformity by an EU instrument by 31/12/2030 will be free to read, at least on request under Regulation 1049/2001. A seller's price does not refute it: EN 18286:2026 has one. Test. Refuted by a refusal standing at that date, which would break my second claim. If none is given one by then, the prediction is untested, not confirmed.
WRONG-1 31/12/2028 mixed By 31/12/2028, at least one professional indemnity insurer publishing wording for solicitors in Australia, or for the layer above the SRA's compulsory minimum in England and Wales, will add an express AI exclusion or condition, and no such condition will name machine-checked evidence. Test. Refuted if none appears, or if one names it.
WRONG-2 31/12/2028 meta, void-able (implied) On 31/12/2028, at least 80% of the 23 no-change predictions counted above will stand confirmed, and fewer than half of the 16 change predictions will. Voids count as unconfirmed. Test. Refuted by either figure failing.
WRONG-3 31/12/2027 change, void-able By 31/12/2027, an audit of at least five of the new chapters, by someone with no stake in the book, will produce at least ten class A corrections. Test. Refuted by a published audit finding fewer. Void if none is done, which would itself be a finding about who wants this checked.
WRONG-4 31/12/2027 change, void-able If the benchmark adapter's bands are tested on a held-out question set by 31/12/2027, the share of wrong answers in the second-highest band will be within ten percentage points of the share of right answers in it. Test. Refuted by a wider gap. Void if the test is not run.
LEDGER-1 30/06/2029 meta, void-able By 30/06/2029, any published scoring of this ledger that covers every entry due by 31/12/2028 will show at least half of its void-able entries due by then ended void or untested. Test. Refuted by a published figure below half. Void if no such scoring is published.
LEDGER-2 31/12/2028 no change By 31/12/2028, nobody with no stake in the book will have published a scoring of ten or more entries of this ledger. Test. Refuted by one such scoring from anyone who meets that description.
LEDGER-3 30/06/2029 meta, void-able By 30/06/2029, in any published scoring of this ledger that covers every entry due by 31/12/2028, refuted entries will outnumber confirmed ones among the change and mixed entries due by then and not void. Test. Refuted by confirmed entries equalling or outnumbering refuted ones. Void if no such scoring is published.
AGENDA-1 31/12/2027 no change By 31/12/2027, no paper or preprint will report a measured covert rate, in bits per proof, between language-model agents through proofs accepted by a proof checker such as Lean, Coq or Isabelle. Test. Refuted by one such measurement, however small the rate.
AGENDA-2 31/12/2028 no change By 31/12/2028, no published evaluation will show a sheaf-derived score beating the minimum of local confidences, with a stated significance test on held-out data, at detecting seeded cross-document inconsistency in legal or regulatory text. Test. Refuted by one such evaluation, with data available for re-running.

About the author: Eduardo Aguilar Pelaez is CTO and co-founder at Legal Engine Ltd. He writes on formal methods, AI agents, and the discipline of building systems that survive being walked away from.