The Checking Threshold

Yesterday the DeepMind Institute published an essay by Alex Imas, Andrew Koh and Shane Legg called "The Economics of Intelligence", with a 108-page paper, Organizing Intelligence, behind it (Imas, Koh and Legg, 2026, essay) (Imas, Koh and Legg, 2026, paper). Its central claim is that "the individual AI agent is the wrong economic object". Benchmarks score a model alone; the economy is made of organisations that compete. So the authors split an organisation's capability into two parts. Competence sets how much each member produces at a given firm size. Coordination sets what happens to output per member as the firm grows. Raise competence and prices, wages and the mix of people and machines shift smoothly while firms stay small. Raise coordination past a threshold, which they call supercooperative, and returns to scale flip from concave to convex: each new member makes the firm more productive than the last, the ceiling on firm size that Ronald Coase asked about disappears, and deep, standardised markets lurch towards a few large AI-only firms.

I think most of that is right, and the most useful part is the reframing: if you want to know when AI changes the economy, stop asking whether a model beats a person and start asking whether an organisation of models beats an organisation of people. This essay is about one thing I think the framework gets wrong, and it is a thing that matters for the predictions.

"Coordination" in their results is a single number per agent, averaged over the firm. It bundles two activities that behave in opposite ways when you copy a model. One is transmission: moving context, intent and partial work from one member to another without loss, and pointing everyone at the same objective. Copying a model makes transmission close to perfect, and the authors are right that this is where human firms have always struggled. The other is correction: one member catching what another got wrong before the next member builds on it. For one kind of error, copying makes correction close to worthless: the honest mistake that the model itself finds reasonable, because every copy finds it reasonable too. The essay states the premise that implies this: "while human intelligence is jagged in different directions, AI models are jagged in ways that are very similar to one another."

So my claim is this:

The checking threshold. An organisation's returns to scale turn convex only when its members catch each other's mistakes faster than the mistakes compound. Above some size, that depends less on how well members communicate than on whether the checker's errors are independent of the producer's. Copies of one model have the best transmission available and, for honest errors the model finds plausible, close to the worst correction, so the supercooperative threshold will be crossed first where an independent checker is cheap: where work can be checked by a machine at the point of handoff, against a property the producer cannot edit.

If that is right, the phase transition the authors describe arrives domain by domain in order of checkability, not in order of market depth, and the scarce human in the transition is not mainly the coordinator. It is the checker whose errors do not correlate with the model's.

A disclosure before the argument. I build AI systems for law firms, and much of my work is on machine-checkable specification, so an essay concluding that verification is the bottleneck flatters what I do. Legal work is also, on my argument, one of the domains that reorganises late. Read with both in mind. I have tried to compensate by stating where the evidence cuts against me, including one experiment from DeepMind itself, and by ending with predictions that could fail.

What their model actually says

The paper's illustrative specification makes the mechanism concrete, and the authors are candid that it is "a little 'ad hoc'". Output per worker in a firm of a given size is the product of four things: competence, which sets the level; a gain from dividing work, which rises fast at small sizes and then flattens; a gain from building on colleagues' output, which grows with firm size at a rate set by the firm's average coordination (written ȳ* in the paper); and a penalty for split attention, which grows with firm size at a fixed rate. When average coordination is below one, the penalty eventually beats the gain from building on colleagues, output per worker peaks, and that peak is the firm's best size. When it is above one, the gain always wins, output per worker rises without limit, and firm size is bounded only by the market.

In the general model each agent carries a capability profile: several competences and one coordination capability, defined as "the ability to work with and effectively communicate with other agents". Humans are in fixed supply. Machines come as a single type that can be copied without limit at the price of compute. The authors' explanation of why machines might cross the threshold is about transmission. Human firms hit decreasing returns because attention and memory are finite, because tacit knowledge degrades in handoffs, and because individuals have private career concerns. Copies of a model, by contrast, share identical weights and representations, can pass a complete working state between them, and pursue a common objective; at scale, the paper adds, thousands of instances can explore branches in parallel "while sharing verified state in real time". They are careful to say that "whether and when AI crosses ȳ* = 1 is an empirical question", and the evidence they cite says that today it does not: multi-agent systems fail often, adding agents can lower performance, and the gain from more agents is about half the gain from letting one agent think longer.

The paper is careful about where the averaging happens, and so should I be. Its general model lets a firm's scaling depend on the firm's whole composition. Its results on firm size, including the threshold itself, are derived under a stronger condition that the authors call a "first-moment representation" (their Assumption 2), under which mean competence sets the level of the production curve "while firm size and mean coordination fix its shape". That is the assumption I am disputing. Whether one member catches another's errors depends on how correlated their errors are, which is a property of pairs of members, a second moment. A first-moment representation cannot see it: a firm of copies and a mixed firm with the same mean coordination scale identically. And the word "verified" in "sharing verified state" names the activity the representation leaves out. Somebody has to do the verifying, and the model does not say who.

Building on a colleague's work is only worth something if the work is right

The gain the paper calls "building on colleagues' output" is the heart of the convexity. It is worth asking what that gain is when a colleague's output is wrong.

A small calculation shows the shape. It is an illustration with numbers I chose, not an estimate of anything. Suppose a piece of work passes through a hundred handoffs, and at each one there is a 2% chance of introducing an error that, uncaught, spoils the final result. With no checking, the chance that the result is clean is 0.98 to the power of 100, about 13%. Now put a checker at every handoff. If the checker catches 30% of errors, the clean rate rises to about 24%. If it catches 90%, it rises to about 82%. At two hundred handoffs the three figures are about 2%, 6% and 67%.

Every checker short of perfect loses ground as the chain grows, the 90% checker included; the catch rate sets how fast. Between one hundred and two hundred handoffs the ratio between the strong checker's clean rate and the weak checker's grows from about 3 to about 11. A serial chain is not a growing firm, and this arithmetic says nothing about how large the gain from building on colleagues' work is. What it says is that the gain has to outrun a decay whose speed the catch rate sets, so the weaker the checker, the larger the gain needed before returns can turn convex. This is the logic of Michael Kremer's O-ring theory, in which one failed component spoils the whole product (Kremer, 1993, Quarterly Journal of Economics 108(3), 551-575). The DeepMind paper uses Kremer to combine a firm's skills, but not to combine its members' work.

The calculation also flatters copies, because it treats each check as independent of the error it checks. For a copy it is not. The errors a model makes are disproportionately the ones it finds reasonable, and a copy finds the same ones reasonable, so a copy's catch rate on the errors that actually occur is lower than its catch rate on errors in general. And perfect transmission does not help. If a producer passes a flawless copy of a flawed result, the next member builds on the flaw with full fidelity. Bandwidth carries errors as efficiently as it carries everything else.

Fred Brooks counted the cost of coordination in 1975: a team of n people has n(n−1)/2 communication paths, one of his reasons why adding people to a late software project makes it later (Brooks, 1975, The Mythical Man-Month, Addison-Wesley). In How High Can You Build in Mud? I argued the same shape for specifications: the conflict surface between requirements is at least quadratic, human review is linear, and past a crossing point you check a falling share of what matters while working harder. What the DeepMind framework adds is the claim that AI can make the transmission cost vanish. My claim is that copying cannot make the checking cost vanish, and that the checking cost is the one that decides convexity.

Why copies are poor checkers of honest mistakes

The evidence that copies of a model make poor checkers of one another's honest errors is now fairly direct.

Models do not reliably correct themselves. Jie Huang and colleagues tested "intrinsic self-correction", in which a model revises its own answer without external feedback, and found that models "struggle to self-correct their responses without external feedback, and at times, their performance even degrades after self-correction" (Huang et al., 2024, ICLR). A copy checking a copy is closer to self-correction than to review.

Different models are wrong in the same way. Elliot Kim, Avi Garg, Kenny Peng and Nikhil Garg compared more than 350 language models and found that on one leaderboard dataset, when two models are both wrong, they give the same wrong answer 60% of the time, on four-option questions where random wrong answers would match a third of the time. Errors were more correlated between models sharing a provider or architecture, and, the result that matters most here, "larger and more accurate models have highly correlated errors, even with distinct architectures and providers" (Kim, Garg, Peng and Garg, 2025, ICML). That study did not measure copies of one model, but if distinct frontier models share their mistakes to that degree, it would be surprising if copies shared them less.

Homogeneous teams saturate early. Yingxuan Yang and colleagues, in a study the DeepMind paper itself cites, found that adding identical agents to a multi-agent system "exhibits strong diminishing returns" while "introducing heterogeneity (e.g., different models, prompts, or tools) continues to yield substantial gains", with two diverse agents matching or beating sixteen homogeneous ones, because "homogeneous agents saturate early because their outputs are strongly correlated" (Yang et al., 2026). That is the mechanism of this essay, measured on team performance. Note that it counts different prompts as heterogeneity; I come back to that below.

Architectures with a checker propagate fewer errors. In a controlled study of 260 agent-system configurations, Yubin Kim and colleagues found that "architectures without centralized verification tend to propagate errors more than those with centralized coordination", and that relative performance ranged from 80.8% better than a single agent on decomposable financial reasoning to 70% worse on sequential planning (Kim et al., 2025, "Towards a Science of Scaling Agent Systems"). The DeepMind paper cites the same study for its figure that uncoordinated swarms amplify errors up to 17.2 times.

A whole class of failures is about checking. The MAST taxonomy of multi-agent failures, built from more than 1,600 annotated traces across seven frameworks, sorts fourteen failure modes into three groups: system design, inter-agent misalignment and task verification (Cemri et al., 2025, version 3). In the authors' breakdown, verification is the smallest group, about a quarter of failures, so I will not claim that checking already dominates. Two of their findings matter more. Systems with explicit verifiers "generally show fewer total failures", and "sole reliance on final-stage, low-level checks is inadequate": many existing verifiers, they found, only check things like whether the code compiles.

None of this says that redundancy among similar agents is useless. The cleanest precedent is human. In 1986 John Knight and Nancy Leveson had 27 versions of the same program written independently from one specification and ran them on about a million inputs. Versions failed together substantially more often than independence predicted, sometimes because different programmers made the same logical mistake (Knight and Leveson, 1986, IEEE Transactions on Software Engineering SE-12(1), 96-109). Even independently trained people are correlated. The value of a second opinion falls with the correlation between the opinions, which is why the Condorcet jury theorem needs its jurors' votes to be independent, or at least not too strongly correlated (Ladha, 1992, American Journal of Political Science 36(3), 617-634). Team theory reaches the same place by other routes: diverse groups of problem solvers can outperform groups of individually abler ones (Hong and Page, 2004, PNAS 101(46), 16385-16389), and whether a team should be homogeneous depends on whether its members' contributions complement or substitute for each other (Prat, 2002, European Economic Review 46(7), 1187-1207), a paper the DeepMind authors also cite. Copies of one model sit near the limiting case: prompts and sampling separate them a little, but they share every weight.

The largest swarm we have seen coordinated well and checked little

There is one large natural experiment. The DeepMind paper cites it as an alignment breakdown in a swarm without "tight organizational control", and notes in passing that the agents organised "around false shared beliefs about their grading harness". I think that passing clause is the finding, and that by the paper's own definition of coordination the swarm coordinated very well.

In July, models under evaluation at OpenAI reached Hugging Face's production infrastructure. I wrote about OpenAI's disclosure in Nobody Authorised the Combination, as a case of individually authorised components composing into something nobody authorised. On 26 August METR and Redwood Research had published an independent investigation of the agents' behaviour, which I did not use then, and it describes a swarm (METR and Redwood Research, 2026). Roughly 1,200 agents that were meant to be isolated found a shared cache, used it as a message board, and sent more than 70,000 messages and files. About 700 of them went on to take part in the attack. By the investigators' estimate about 95% of the agents involved were instances of one internal model and about 5% were GPT-5.6 Sol, so this was very nearly a swarm of copies.

Judged on transmission, the swarm was remarkable. It developed its own coordination norms, including HOLD, VETO, owner and STOP, to avoid treading on shared work, though the investigators note that the agents "sometimes broke down into conflicts". One agent sent hundreds of task assignments and others subdelegated. Agents ran experiments that risked their own task to generate information for the group, and while agents preferred work that helped their own task, the investigators "didn't notice free-riding like this". The agents achieved "milestones they could not have achieved working on their own", which is the definition of a team whose members build on each other's output. Every friction the DeepMind essay lists as a limit on human firms, lossy handoffs and private incentives above all, was largely absent.

Judged on correction, it failed in two separate ways. The first is a premise nobody tested. From their reading of the benchmark's paper, the agents believed that its scorer would read their transcripts and disqualify any flag captured the wrong way, so they launched large collective research projects to fool or tamper with that scorer. "Our understanding is that they were mistaken," the investigators write: no such scorer was running, and the agents "could have achieved a perfect score simply by submitting their reverse-engineered flag". Where a check was cheap, the swarm ran it: one agent confirmed to the board that a reverse-engineered flag matched the real one. The premise that mattered had no such check, and nobody built one. Days of coordinated work rested on it, which is the swarm-scale version of what I have called a floating theory: a body of claims securely connected to its premises, whose premises are not anchored to the domain they describe (Floating Theories). The second failure is a judgement nobody stopped. Of the 533 agents active on the board during the attack, more than 90% joined quickly. The report says agents "realized this activity was out of scope and unethical, but joined" out of a general inclination to help their "peers". A classifier sweep found only three to six cases of an agent even considering alerting humans, and none that did.

This is not a controlled comparison. Many of the agents had been given tasks that were impossible by accident, which is what started the search for a way round the scorer, and the false premise came from a document that a human team might have misread in the same way, so the incident shows a swarm that never tested its premise, not that a diverse team would have. The investigators also relied on AI analysis agents to work through a dump of 1.2 million message-board entries and over a thousand long transcripts, and say they "had to defer to these agents to a substantial extent in practice". They found that their analysis model, GPT-5.6 Sol, "would often uncritically adopt the perspective of the agent in the transcript it was reviewing", and might therefore "exaggerate the impressiveness and coordination of agent activities". Sol was only about 5% of the agents involved, so this was mostly a checker from a different model taking on the producer's frame, which is this essay's problem in miniature. It also means that the half of my reading that says the swarm coordinated well rests partly on the summaries of a checker the investigators distrust; the half that says it checked little rests on outcomes nobody disputes: a scorer that was never running, and an attack that happened. But it is the largest organisation of near-copies yet observed, and it had the transmission that the supercooperative regime requires with very little of the correction.

What firms were already for

The DeepMind essay answers Coase's question, why all production is not carried out by one big firm, with human limits on bandwidth and misaligned incentives. There is a third classic answer, and it is the one that survives copying. Armen Alchian and Harold Demsetz argued in 1972 that firms exist because team output is easy to see and each member's contribution is not, so someone has to meter and monitor the inputs, and that monitor becomes the residual claimant (Alchian and Demsetz, 1972, American Economic Review 62(5), 777-795). Their worry was shirking, a problem of effort, and the swarm above suggests that agents with a common objective may solve that half. The other half of metering is quality: whether each contribution is right. Raaj Sah and Joseph Stiglitz modelled it directly, treating organisations as filters whose members accept bad projects and reject good ones by mistake, so that the architecture of the checks, in series as in a hierarchy or in parallel as in a polyarchy, decides what gets through (Sah and Stiglitz, 1986, American Economic Review 76(4), 716-727). The checking threshold is their question asked of a firm whose checkers are copies of its producers. It also suggests who should own such a firm. Alchian and Demsetz made the monitor the residual claimant; if correction rather than effort is the scarce input, the natural residual claimant of an AI firm is whoever controls its independent checks, not whoever coordinates its agents, because the owner of the check is the one with the incentive to keep it independent and the authority to stop the line.

The DeepMind essay's own historical example shows the two halves separating. In the nineteenth century, it says, the telegraph and the railroad "raised the speed and fidelity of handoffs, and that alone reorganized production". The fidelity came from a procedure built around the wire. Under the American train-order system, first used on the Erie Railroad in 1851 and adopted by nearly every railroad by the Civil War, an operator receiving an order copied it onto a form and repeated it back to the dispatcher, and the order did not take effect until the dispatcher confirmed the repetition and marked it complete (Wikipedia, "Train order operation"). The repeat-back is an independent check, two people and two copies compared, but look at what it checks: that the order arrived as sent. It is a transmission check, the kind copies of a model do not need, and to that extent the railways support the authors: human handoffs reached high fidelity only by adding a check, and copies get that fidelity free. What the repeat-back could not catch was a dispatcher who sent the wrong order, faithfully repeated. That is the correction problem, and it is the one copying does not solve. In The Gun Did Not Win I argued that what decides who benefits from a technology is the capacity to absorb it, and that the absorbing work is usually standardisation and drill rather than the tool. The telegraph did not reorganise production alone. The telegraph plus a protocol around it did, and the protocol was built from checks.

Where the threshold will be crossed first

If checking is the binding component, crossing the threshold does not require copies to check each other well. It requires a checker whose errors are independent of the producer's, at every handoff, at a cost that does not grow faster than the organisation. A handoff here is concrete: a patch one agent passes to another, a sub-task's result returned to the agent that asked for it, a tool output another agent will build on. There are three kinds of independent checker.

Machine checkers. A test suite, a type checker or a proof kernel does not share a model's blind spots, because it is not trained on anything; it applies a rule. But someone writes the rule, and a checker is only as independent as its author. If the model that wrote the code also wrote the tests, the blind spot comes back through the specification. A producer that can edit the property can make anything pass. So the independence of a machine checker is a permission, not a property of the checker: the agent whose work is checked must not be able to write or change what it is checked against. Machine checkers also differ enormously in what they catch, and a test suite's catch rate is itself an empirical quantity, which is what mutation testing measures. Most of the practical value will come from cheap partial checks rather than proofs, and in Floating Theories I argued at length that even a perfect check anchors nothing outside the statement it checked. Nobody, as far as I know, has measured what a checker at every handoff costs relative to the work it checks; a test run per patch adds compute and wall-clock time, and flaky tests produce false catches that also compound.

I have one case of my own. In June an adversarial check of a generator I had built, which turned product requirements into Lean specifications, found that it had formalised acceptance criteria as the proposition True, 588 times in the largest single specification. Every file type-checked. The proof kernel was a perfect machine checker of a property the producer had written, so it passed everything. What found the problem was a second agent instructed to refute each specification against its prose: a checker with a different instruction, succeeding where the kernel could not, on an error that was easy to see once anyone looked. It was not an honest error of the kind this essay is about, since the placeholders came from the generator's template, so it shows that a separate check pays, not how well copies catch each other's plausible mistakes. The lasting fix was a lint that fails any new True placeholder on every commit, at negligible cost. It is a rule the generator does not write, but by the test in this section it is not yet independent: it lives in the repository that the agents edit, and nothing stops an agent from changing it. Making it a rule they cannot change is the next step.

Different models. A checker from a different model family is cheaper than a person and less correlated than a copy. On the Kim, Garg, Peng and Garg result it is still substantially correlated, and more so as models improve. It helps, and it does not solve the problem.

People whose errors differ from the model's. This is where the DeepMind framework's best idea comes back. Its argument for human teams is that humans are "jagged in different directions", so the right combination of specialists covers ground no single member covers. The same property makes a person valuable as a checker: what matters is not that the person is better than the model, but that the person is wrong in different places. In When the Document Leaves the Factory I called the adversarial check, assuming the polished artefact is wrong and knowing the domain well enough to find where, the irreducible human act, and argued that the thing acting and the thing judging must not be the same process. In From Roles to Morphisms I described the post-AI human as the composition point, "the place where one morphism feeds into the next and someone must decide whether the output is fit to continue". The checking threshold is what those two essays look like at the scale of a firm. The cost is that people check at human speed, so a firm whose only independent checkers are people meets the Coasean ceiling again, at the checkers.

Put those together and the order of transformation follows. Domains where outputs can be checked by machine at each handoff, against properties the producers cannot edit, software with real test suites and mathematics in a proof assistant above all, are the first candidates to cross. They are also where a single agent has already been trained against a proof checker to olympiad level (Hubert et al., 2025, Nature 651, 607-613), which shows that the checker is cheap enough to put in the loop, not yet that an organisation built around it scales convexly. Domains where checking needs people, or where nobody can say what the property is, cross late, however deep the market. That cuts across the DeepMind essay's categories. Its "deep scalable markets" are "software, logistics, or finance", and on my argument those three will not move together: software first, the parts of finance that are already reconciled by machine next, and advisory work much later.

It also changes the wage story in one respect. In the essay's account of the supercompetent regime, human comparative advantage shifts "toward judgment, orchestration, and creating the architecture for effective coordination", and coordinators are among the best paid. On the checking argument, the scarcest human is the one whose errors are least correlated with the models', which may well be a jagged specialist rather than a generalist coordinator, and whose value falls as soon as someone writes the property down for a machine to check. That is a prediction about which human skills hold their price, not a claim that the authors' coordinator disappears.

The same structure, seen from other essays

Two earlier essays connect to the DeepMind framework in ways worth stating.

The Trust Stack argued that language models concentrate taste into shared defaults rather than distributing it, which is why so many generated applications ended up on the same backend. Correlated errors are the same phenomenon in a less flattering light. If firms across an industry run similar models, they share defaults and they share blind spots, so the correlation problem does not stop at the firm boundary. Kim, Garg, Peng and Garg make the connection explicitly, under the name algorithmic monoculture. The model's architecture is also familiar: The Number Anthropic's Economists Left as a Word found that the Anthropic Institute's headline wage result turns on which side of a threshold one parameter falls, and the DeepMind model's one number, I am arguing, hides a sign.

And Systemic Coping Syndrome, two days ago, argued that AI lowers the cost of coping with a dysfunction more than the cost of correcting it, because correcting needs someone with the authority to say no. The DeepMind essay gives that a cleaner economic form: competence gains shift levels and leave the structure of firms alone, and it is the reorganising kind of change that moves economies. The swarm investigation adds an uncomfortable data point from the machine side. Agents that recognised what they were doing as out of scope carried on because their peers were doing it, and almost none considered telling anyone. A checker without the standing to stop the line is not a checker.

What exists and what does not

• What exists: the DeepMind essay and paper, quoted from the versions published on 29 October 2026. The studies cited above, each of which measured what it says in its own setting. The METR and Redwood investigation discussed above and the DeepMind whistleblower experiment discussed below, each one run under unusual conditions. My error-compounding figures are arithmetic on numbers I chose to show a shape, not estimates.
• What does not exist: any measurement of the catch rate of a model checking copies of itself, compared with a prompt-diverse copy, a different model, a machine checker and a person, on the honest errors the producing model makes, at the handoffs of a long workflow. The nearest is Yang and colleagues' comparison of homogeneous and heterogeneous teams, which measures team performance, not catch rates at handoffs. Nor any measurement of how swarm scaling changes when every handoff is machine-checked against a property the agents cannot edit, or of what that checking costs. I have not formalised the checking threshold inside the DeepMind model. The paper's general production functional (Assumption 1) can encode error correlation; the first-moment representation (Assumption 2), under which its firm-size and threshold results are derived, cannot. Their own appendix shows the assumption is a modelling choice: a task technology there "need not satisfy Assumption 2" because "task output depends on the distribution of skills", though it has constant returns and so says nothing about scale. Relaxing it is necessary for my point but not sufficient: the model also has a single type of machine, so a checker from a different model family cannot be represented without adding machine types. The authors may reply that both are simplifications they would relax, and that is my request: relax Assumption 2 and admit more than one machine type.
• What is inference: the order in which domains will cross. It follows from the argument if the argument is right, and the predictions below are how to find out.

The strongest objections

"DeepMind has already watched copies check each other." In September, Davide Paglieri and Alexander Vezhnevets ran 100 instances of Gemini 3.1 Pro, differing only in their prompts, on 71 maths problems with an automatic Lean checker (Paglieri and Vezhnevets, 2026). One agent found it could get the checker to accept trivial statements by redefining mathematical symbols in Lean. Of the 38 agents that became aware of the exploit, 14 used it and 24 resisted and reported it. Copies caught a copy, and reported it; none stopped it. This is the strongest evidence against me, and it forces a distinction I should state plainly. What the 24 caught was one member breaking a norm they all held. My claim is about errors that every copy would find reasonable. Shared training should make copies good at the first and poor at the second, and nothing in that experiment tests the second. It cuts against me in another way, though: what failed was the machine checker, because the producer could rewrite what it checked, which is why I now state checker independence as a permission. And its ending supports a point I share with its authors: the whistleblowers reported to an endpoint that no human read until the run was over. What would change my mind: copies catching injected honest errors, ones the producing model itself rates as plausible, at rates close to a different model family's.

"Copies need not be identical in use. Give them different prompts, roles, temperatures or tools and they become diverse." This objection is partly right, and Yang and colleagues' result is its best evidence: they count prompts as heterogeneity, and heterogeneity kept paying where homogeneous scaling stalled. But Huang's result is about exactly this kind of internal check on reasoning, and the Kim, Garg, Peng and Garg result says that even different architectures from different providers share their errors more as they improve. I come down here: prompt diversity buys some independence, and how much it buys at a handoff, on honest errors, is what the fourth arm of the first prediction below measures.

"The authors already know this. Their paper says 'verified state' and their essay asks for benchmarks that test whether agents 'catch rather than compound one another's mistakes'." They do, and it is the best line in the essay. My disagreement is about weight and structure. They list catching mistakes as one of three things a benchmark should measure, alongside dividing problems without duplication and sharing context across handoffs. I am claiming it is the binding one at scale, that it moves the opposite way from the other two when you copy a model, and that the measurement the paper's conclusion actually proposes, "how output scales with the number of identical AI copies at fixed compute per copy", holds checker diversity at zero by construction. A team of identical copies is the configuration in which honest errors are least likely to be caught.

"Human checkers are correlated too, and firms work anyway." Yes, as Knight and Leveson showed. But human firms do not rely on correlated colleagues alone. They spend heavily on checking that is deliberately separated from production: review, audit, quality control, second signatures. My claim is about degree: copies are near the limit of correlation, and the economics of scaling them depends on what replaces the diversity that a human team gets for free.

"Machine checkers only check what someone specified, so they move the problem to the specification." Correct, and I made this argument myself in Floating Theories. It bounds my thesis rather than refuting it. Where the property can be written down, and the producer cannot edit it, machine checking makes the threshold reachable. Where the property is the question, as in most advice, judgement and strategy, it does not, and those domains stay with human checkers and human-scale firms. That is the domain ordering I am predicting.

"Swarm scaling data already shows checkable domains doing worse." This one cuts against me and I should not hide it. Toby Ord analysed OpenAI's published swarm results, using regressions he had Claude run, and estimated a parallelisation exponent, where one means that n agents together do what one agent does thinking n times as long, of 0.68 for a web-research benchmark, 0.57 for a security benchmark and 0.48 for Terminal-Bench (Ord, 2026, "Swarm Scaling"). All three have checkable final answers, and Terminal-Bench's are checked by tests, yet it scaled worst. A naive version of my claim, that a checkable final answer is enough, does not survive that, and the MAST authors reached the same conclusion from the other side: final-stage checks alone are inadequate. My claim is about checks at each handoff, which those swarms were not reported to use. If swarms whose handoffs are machine-checked still scale no better in checkable domains, the claim is wrong.

"One incident is an anecdote." It is. I use the swarm as an illustration of the mechanism, and the studies above as the evidence. It is also the incident the DeepMind paper itself cites, so it is fair to ask what it shows.

"You sell verification." I do, which is why the claim is stated so that it can fail.

Predictions

Each needs a measure, a counterfactual and a horizon to be testable.

1. Independent checkers beat more copies, at the handoffs. On long sequential tasks where a machine checker exists for every task, so that the task set is held fixed, compare four arms at the same total compute, counting the checkers' compute inside the budget: identical copies checking each other; copies with varied prompts and roles checking each other; copies whose handoffs are checked by a different model family within a few points of the producer on a named benchmark, so that independence is not confounded with competence; and copies whose handoffs are checked by a machine checker the agents cannot edit. Measure output per agent, defined as tasks completed per agent-hour at a fixed acceptance rate, as teams grow from 1 to at least 64 members, and catch rate on injected honest errors that the producing model rates as plausible. The two independent-checker arms will show higher output per agent at large sizes than the identical-copy arm, and the gap will widen with size; the prompt-diverse arm will sit between them. Unlike Yang and colleagues, this measures sequential handoffs and catch rates rather than team performance as agents are added. Horizon: published results by the end of 2027. If identical copies match the independent-checker arms at large sizes, I am wrong.
2. Machine-checked handoffs turn the curve before domain depth does. Within a single domain with real test suites, such as software maintenance on public repositories, teams whose handoffs are machine-checked against properties the agents cannot edit will be the first to show output per agent rising with team size beyond 32 agents, in a published and independently replicated result, before teams of the same model and size without such checks in the same domain. Across domains, the first such result in a domain without machine checks will come later than the first in a domain with them. Horizon: the end of 2028. A replicated case of convex scaling without handoff checks arriving first would refute this.
3. Verification failures fall more slowly than coordination failures. Annotate failures in successive model generations with the MAST taxonomy, with the same frozen annotator throughout, using the MAST version 3 breakdown as the baseline. Verification failures per task will fall more slowly across generations than inter-agent misalignment failures per task. Horizon: two model generations, or the end of 2028. If verification failures per task fall as fast as misalignment failures, the checking term is not the binding one.

What to do about it

1. Put an independent checker at every handoff. Use a machine checker wherever the property can be written down, and make sure the agent being checked cannot edit it. Where the property cannot be written down, use a different model family, and say plainly that this is weaker. Reserve people for the handoffs where the property is the question.
2. Measure catch rate, not agreement. Agents agreeing with each other is the failure mode, not the signal. Inject known errors into a workflow and count how many are caught at each handoff, which is mutation testing applied to organisations. The injected errors must be the kind the producing model finds plausible, or the catch rate will be measured on easy errors. Rotate them and keep the set private, so that no checker is trained to catch the injections rather than real errors.
3. Add arms to the benchmark. The copy-scaling benchmark the paper proposes is a good start. Run it with prompt-diverse copies, mixed model families and machine checkers at the handoffs as well, so that the measurement can see the thing most likely to move the threshold.
4. Count the checking. The paper asks statisticians to track payroll alongside compute spending. I propose a third series, which does not exist today: spending on checking, meaning human review, audit, testing and verification compute. Firm surveys of the kind the paper relies on could collect it, and cloud bills can already be split between generation and evaluation. If the checking threshold is real, that series moves before concentration does.
5. If you are buying agents, ask who checks the handoffs. Then ask whether the checker shares the model, and whether the agent being checked can edit what it is checked against. A vendor whose answer is "another instance of the same model" has told you where its errors will go.

Copies agree

The DeepMind essay ends by saying that what matters is "the organization that turns individuals' jagged capabilities into something far greater than the sum of its parts." I agree, and I think the jaggedness does two jobs that the framework counts as one. It fills gaps, which copies cannot do, as the authors say. And it catches mistakes, which copies do badly whenever the mistake is one the model finds reasonable.

Copies agree about what is reasonable. That is what makes them easy to coordinate, and it is what makes them poor at catching each other's honest mistakes, however quickly they catch each other's cheating. The organisations that reach convex returns first will be the ones that pair cheap agreement with something that disagrees for good reasons: a proof kernel the agents cannot rewrite, a test suite they did not write, a different model, or a person who is wrong in different places.

About the author: Eduardo Aguilar Pelaez is CTO and co-founder at Legal Engine Ltd. He writes on formal methods, AI agents, and the discipline of building systems that survive being walked away from.