SharePlane / Research Thesis / Candidate

The Proof Arrived Before the Understanding

Machine discovery is accelerating. Our ability to establish what those discoveries mean, and whether we can trust them, is becoming a very different problem.

Tony Malott · October 8, 2026

Machine discovery is accelerating. Our ability to establish what those discoveries mean, and whether we can trust them, is becoming a very different problem.

Tony Malott | October 8, 2026

I've been watching the latest developments in artificial intelligence with more than a little fascination, particularly as these systems become capable of doing meaningful work without someone standing over them directing every move. Most of my attention has been on the engineering side, figuring out how to coordinate agents, give them access to real machinery, and establish enough control that they can operate autonomously without creating a spectacular mess. There's a considerable difference between watching an agent produce a clever answer and watching a thousand of them accomplish something useful together. I'm increasingly convinced that the coordination of intelligence is becoming as important as the intelligence of any individual model.

Then OpenAI released its latest mathematical research, and I found myself thinking about the same problem from an entirely different direction.

On October 6, OpenAI published a collection of 722 mathematical manuscripts organized into 372 research families, produced using an internal frontier model. These weren't simply mathematical exercises designed to demonstrate benchmark performance. The collection included proposed advances on longstanding open problems across several disciplines, accompanied by research artifacts and, in many cases, formal proofs that could be checked using Lean. A month earlier, the company had announced a proposed solution to the Navier–Stokes Millennium Prize problem, describing an effort involving approximately 10,000 concurrent agents in the successful research group and 130 billion output tokens devoted to that investigation.

I find the scale of that achievement extraordinary, and I'm not particularly interested in diminishing it simply because the work came from a machine. If these systems can help mathematicians solve problems that have resisted decades of human effort, we should be celebrating that capability and figuring out how to make it useful. But the announcement also raised a question that I've been encountering in my own work with autonomous systems: what happens when the machinery becomes capable of generating results faster than we can establish what those results actually mean?

I suspect we're approaching a point where producing candidate knowledge becomes relatively inexpensive, while establishing its validity, interpreting its significance, and deciding what authority it deserves remains difficult. That distinction changes how we should think about the architecture of research, and perhaps the architecture of intelligence itself.

The discoveries and the corrections

What happened the day after OpenAI's announcement may be nearly as instructive as the announcement itself. On October 7, the company withdrew three manuscripts after identifying a sign error that invalidated an argument involving algebraic geometry and affected two dependent results. Fourteen other manuscripts were revised to address proof repairs, corrected statements, clarified assumptions, and related issues. The collection dropped to 719 manuscripts, with approximately 42 percent of the top-line results accompanied by Lean formalizations.

I wouldn't interpret those corrections as evidence that the research program had failed. Mathematics advances through conjecture, proof, criticism, and revision, and the fact that OpenAI documented the changes and preserved its revision history is encouraging. There's a legitimate difference between a research system that acknowledges defects and one that quietly replaces its conclusions while continuing to project confidence. The interesting question is whether these correction mechanisms can keep pace when the volume and complexity of machine-generated research increase dramatically.

Consider what happens when an unverified result becomes the foundation for several subsequent discoveries. A second system extends the argument, a third identifies an application, and a fourth incorporates the conclusion into a broader theory. If the original result contains an error, the problem isn't confined to one manuscript. We now have a network of dependent claims whose validity may need to be reconsidered, and the complexity of that reassessment grows with every additional dependency.

This is familiar territory to anyone who has spent enough time building and maintaining large systems. We accumulate technical debt when implementation moves faster than our ability to maintain the integrity of what we're building. I think something comparable can happen with knowledge, particularly when autonomous systems begin generating and combining research at industrial scale. We could call it epistemic debt: an expanding inventory of claims whose assumptions, verification status, dependencies, and limitations haven't been adequately established.

The phrase matters less than the operational problem. A result doesn't become more reliable because a hundred downstream arguments depend on it, and repeated use can create an appearance of legitimacy that is difficult to unwind. Scientific communities have dealt with erroneous foundations for centuries, but they haven't previously had to contend with this degree of automated production. We should be thinking about the verification and correction machinery now, before the amount of accumulated work makes reconstruction prohibitively expensive.

When the proof and the problem aren't the same thing

The Navier–Stokes announcement introduced another dimension that I find particularly interesting. OpenAI published a formalization in Lean, a proof assistant that can mechanically check whether a mathematical derivation follows from its stated definitions, assumptions, and logical rules. That's considerably more meaningful than having another language model read an argument and announce that it looks correct. Formal verification can eliminate entire categories of reasoning errors that might otherwise escape inspection.

But the formal checker can only establish properties of the statement and proof it has been given. It doesn't automatically establish that the formalized statement faithfully represents the original mathematical problem, or that every assumption introduced during translation is justified.

On October 6, mathematicians Alexander Bastounis, Fabian Circelli, and Anders C. Hansen published a critique examining precisely this issue in OpenAI's Navier–Stokes work. They argued that the Lean formalization did not faithfully correspond to the natural-language proof. Their argument is itself subject to independent scrutiny, and it would be premature to treat the critique as the final determination of the original result. Nevertheless, the concern it identifies is fundamental: a mechanically valid derivation can still leave unanswered whether the correct proposition was formalized in the first place.

I've encountered a less mathematically sophisticated version of this problem repeatedly in systems engineering. You can build a technically excellent implementation against the wrong requirements, test it against an equally incorrect specification, and receive a beautiful collection of passing results. Everything looks healthy until someone attempts to use the system for its intended purpose. At that point, the distinction between satisfying the test and satisfying the requirement becomes painfully obvious.

I recently encountered exactly that kind of disconnect in our own publication machinery. We had spent considerable effort qualifying the technical paths, only to discover that the experience reaching user acceptance testing was using the wrong presentation wrapper. The machinery was performing work, and portions of it were passing their checks, but we hadn't adequately established that the resulting experience corresponded to what we actually intended to deliver. More automation wouldn't have corrected the underlying misunderstanding. It might simply have produced the wrong result more efficiently.

A mathematical proof is obviously not the same thing as a publishing pipeline, and I wouldn't pretend that the difficulty of formalizing advanced mathematics is equivalent to validating software requirements. The common problem is the boundary between an intended meaning and the representation a machine is asked to evaluate. That boundary deserves independent attention, particularly when the systems responsible for generating and checking the work are themselves probabilistic.

Formal verification is an extraordinary tool, but it cannot be expected to compensate for every defect in the process that defines what should be verified. We need to preserve the distinction between a statement being correctly proved and the statement being the one we intended to establish.

Metacognition has to become operational

This brings me to metacognition, a subject I've been exploring more seriously as the autonomy and complexity of agent systems increase. In humans, we generally associate metacognition with the ability to examine our own thinking, recognize uncertainty, reconsider assumptions, and notice when our methods of reasoning may be leading us astray. Applying that concept to artificial intelligence is tempting, but I think we need to be considerably more precise about what we're claiming.

A language model can produce a sophisticated explanation of its limitations, acknowledge uncertainty, and describe possible errors in its reasoning. None of that establishes that it has reliably identified the errors that actually matter. It may be producing an excellent description of epistemic humility without demonstrating the underlying capability. The difference becomes particularly important when that model is operating autonomously and its conclusions can trigger real actions.

I've also become skeptical of treating agreement among multiple agents as inherently strong evidence. Ten thousand agents running concurrently may provide extraordinary computational breadth, but concurrency doesn't guarantee independence. If the agents share a model, training history, source material, or interpretation of the problem, their errors can be correlated. Even using multiple model families doesn't eliminate that possibility when the models rely on the same assumptions or derive their conclusions from overlapping evidence.

The question I want our systems to answer is more demanding than whether several agents reached the same conclusion. I want to know whether they arrived there independently, what evidence could have changed their minds, which alternative explanations they considered, and whether the evaluation was genuinely capable of exposing an error in the original reasoning. A system designed primarily to confirm its own conclusions may become extremely efficient at producing agreement without becoming more reliable.

What interests me about engineering metacognition is that we can define observable behaviors without pretending to settle the philosophical question of machine consciousness. Suppose an agent encounters evidence that contradicts a conclusion it previously accepted. Can it identify the contradiction, determine which assumptions are affected, revisit dependent conclusions, and change its planned actions accordingly? If an independent verifier challenges a result, can the system preserve that disagreement rather than allowing a summarization step to erase it? Can it distinguish uncertainty about a factual claim from uncertainty about whether the claim was formulated correctly?

These are capabilities we can build experiments around. We can introduce false assumptions, conflicting evidence, contaminated sources, and dependencies that become invalid after an upstream correction. We can measure whether the system detects the problem, whether it responds appropriately, and whether its behavior improves across controlled conditions. It won't resolve every philosophical question about what the machine understands internally, but it would give us something considerably more useful than another model claiming to be self-aware.

There is an important operational consequence here. Recognizing uncertainty only matters if that recognition changes what the system is permitted to do. An autonomous agent that announces it might be wrong and then executes an irreversible action anyway hasn't demonstrated much useful self-governance. In consequential environments, uncertainty needs to influence authority, execution boundaries, and the evidence required before proceeding.

This is where I think metacognition moves from an interesting research topic into a genuine engineering discipline.

The authority problem

As frontier models become more capable, I expect some forms of verification to become increasingly automated as well. Mathematics is particularly promising because formal proof systems can check explicit logical derivations without relying on human intuition about whether an argument seems reasonable. It's entirely plausible that machines will eventually produce and verify significant mathematical results that very few humans can understand in their entirety.

I don't find that prospect particularly troubling. Human mathematicians already rely on bodies of knowledge that no single individual can independently reconstruct, and computer-assisted proofs have been part of the discipline for decades. We shouldn't impose an arbitrary requirement for human comprehension simply to preserve the appearance that people remain intellectually indispensable to every calculation.

But accepting machine-assisted verification doesn't mean granting unrestricted authority to the system that generated the result. There are still important questions about whether the specification captures the intended problem, whether the assumptions are legitimate, whether the result has been independently evaluated, and what consequences follow from treating the claim as established. Those questions aren't necessarily resolved by making the underlying model more capable.

I think this is also where the growing concentration of frontier AI capability deserves serious scrutiny. If a small number of organizations possess systems capable of producing enormous amounts of advanced research, those organizations will influence which questions receive attention, which methods are available, and how much of the underlying evidence can be independently examined. That doesn't give them ownership of mathematical truth, but it can give them considerable influence over the practical direction of scientific inquiry.

The independent Advisory Group on Mathematics and Artificial Intelligence has already emphasized the importance of preserving researchers' autonomy and improving how AI-generated mathematical results enter the broader scientific community. I agree with that concern, although I don't believe the solution is to constrain capable systems until existing institutions feel comfortable with the pace of change. We should be increasing access to rigorous verification, independent evaluation, and reproducible research rather than treating the capabilities themselves as something that needs to be artificially suppressed.

There's also a danger in constructing elaborate verification machinery that creates the appearance of independence without actually providing it. If the same underlying assumptions influence the researcher, the reviewer, and the final adjudicator, adding more layers may increase complexity without materially improving reliability. We should demand evidence that the verification architecture detects defects the original system would otherwise miss. Anything less risks turning scientific assurance into an expensive ceremony.

What the bottleneck actually is

I've written previously about how the bottleneck in software engineering shifts when implementation becomes abundant. As autonomous systems become capable of generating, modifying, testing, and deploying increasingly complex software, the limiting factors move toward judgment, architecture, integration, verification, and deciding what should be allowed to operate.

Watching OpenAI's mathematical research unfold, I suspect something similar is beginning to happen in scientific discovery. We may be approaching a period when generating plausible solutions to extremely difficult problems is no longer the principal constraint. Establishing their correctness, understanding their relationships, and determining which conclusions deserve confidence could become the more demanding work.

There is an obvious counterargument, and it's worth taking seriously. Perhaps the same advances that make mathematical discovery dramatically faster will also make verification so capable and inexpensive that the entire problem largely resolves itself. I hope that's true for substantial portions of formal mathematics. If we can reliably automate both the construction of proofs and the independent checking of their correspondence to the intended questions, we should pursue that capability aggressively.

I remain less convinced that the same approach will generalize cleanly across every domain where autonomous intelligence operates. Mathematical statements, scientific hypotheses, engineering decisions, clinical judgments, and production changes don't share a single standard of evidence. The consequences of error differ, and some decisions involve values and responsibilities that aren't reducible to formal proof. Our governance machinery needs to recognize those differences rather than applying the appearance of mathematical certainty to problems that don't possess the same structure.

For my own work, the practical direction is becoming clearer. I want autonomous systems to explore aggressively, use different models when useful, challenge one another's conclusions, and take advantage of every legitimate computational resource available to them. But I also want their claims to carry enough provenance, dependency information, and verification history that we can reconstruct how a conclusion was reached and determine what needs to change when it turns out to be wrong.

That is a considerably harder problem than orchestrating a large number of agents, although the orchestration itself is hardly trivial. It requires separating the intelligence that generates conclusions from the authority that permits those conclusions to influence consequential actions. It also means building systems that can preserve uncertainty and disagreement without becoming paralyzed by either of them.

What OpenAI has demonstrated is impressive enough that I think we should resist the temptation to turn the conversation into another simplistic contest between human and machine intelligence. The research deserves serious mathematical examination, and the outstanding questions about its validity deserve equally serious attention. We may eventually look back on this period as the beginning of an extraordinary expansion in our ability to discover things that previously seemed beyond reach.

I hope we do. I also hope we recognize that producing discoveries and establishing trustworthy knowledge are different accomplishments, because the distinction is going to matter far beyond mathematics.

I've spent years trying to make systems more capable, more autonomous, and less dependent on constant human intervention. I don't intend to reverse that direction now. But the more capable the machinery becomes, the more interested I am in whether it can recognize the boundaries of what it has actually established, and whether those boundaries have any meaningful effect on what it does next.

That's where I think the next important work is.

Not in teaching machines to sound more certain about their discoveries, but in engineering systems that can demonstrate when that certainty is justified, and behave appropriately when it isn't.


Sources and evidence

  1. OpenAI, Sharing AI Progress in Mathematics, October 6, 2026.
  2. OpenAI Mathematical Research Repository, including the catalog, Lean formalizations, and research summaries.
  3. OpenAI Research Revision History, October 7, 2026.
  4. OpenAI, On the Navier–Stokes Millennium Prize Problem, September 8, 2026.
  5. Bastounis, Circelli, and Hansen, Navier–Stokes Lost in Translation, October 6, 2026.
  6. Advisory Group on Mathematics and Artificial Intelligence, October 6, 2026.

The distinction between mathematical claims and independently established results is intentional. The proposed concepts of epistemic debt and operational metacognition are analytical interpretations, not conclusions attributed to OpenAI or the cited researchers.