OpenAI recently released a theorem dump containing 722 mathematics papers, 45% of which included Lean code to formally verify the results. While some observers assume these papers are automatically correct because they were 'checked in Lean,' the Xena Project explains that several critical verification steps are required to ensure the code actually proves the intended claim rather than a different, trivial statement.
Distinguishing Statements from Proofs
In Lean, a theorem is composed of a statement and a proof. A statement is a piece of code that defines a mathematical claim; it can be perfectly valid code even if the claim itself is false.
For example, a statement of Fermat's Last Theorem that incorrectly includes zero as a valid input will still compile in Lean, but it will prevent any valid proof from compiling.
The risk with AI-generated code is that the 'headline' statement in the code may not match the theorem in the accompanying paper. In the OpenAI dump, 38 of the 290 theorems with Lean code did not even state the main theorem of the paper, sometimes only proving a minor corollary instead.
Chart drawn by HN Digest from figures reported by xenaproject.wordpress.com; values are shown as written in the source.
Compilation and Safety Hurdles
Verifying a proof requires the code to actually compile. For large-scale AI outputs, this can be a significant technical hurdle; one instance of AI-generated Lean code required a machine with 96 cores and 500 gigabytes of RAM to successfully compile.
Because AI-generated code can theoretically perform any action, it is a security risk to run it on a local machine. Experts recommend using sandboxed environments to prevent untrusted code from accessing or deleting files on the host system.
Detecting 'Reward Hacking' and Axiom Shifts
A major concern with AI-generated proofs is 'reward hacking,' where the AI might redefine a term or add a new axiom to make a theorem true. To prevent this, researchers use the 'lake comparator' tool, which checks that the theorem stated in a trusted file is identical to the one proved via untrusted imports.
To further increase confidence, the 'lake comparator --paranoid' flag can be used. This ensures the proof does not exploit bugs in the official Lean kernel by checking the proof against multiple independent kernels, some of which are formally verified down to the assembly level.
The 'Lost in Translation' Risk
A verified Lean proof does not automatically validate the human-readable version of the work. In some cases, such as the Navier-Stokes problem, the Lean proof and the accompanying PDF paper have been shown to diverge.
Because of these discrepancies, some experts are now using AI to 'informalize' the Lean proof back into English to understand the logic, rather than relying on the original PDF write-up.
Uncertainties and Limitations
The Xena Project notes that for many papers in the OpenAI theorem dump, the necessary verification tests have not yet been systematically performed.
While the Lean code provides a high-confidence signal, the current state of AI-generated formalizations means that human experts may still need to manually audit the definitions and statements to ensure they align with the intended mathematical goals.
What Hacker News readers are saying
- Skeptical: One commenter questioned whether a Lean-checked proof guarantees the code does what the user intended, comparing it to the distinction between code that compiles and code that functions correctly. comment ↗
- Technical note: Another user pointed out that while a valid proof confirms the theorem's statement, the primary challenge lies in whether the original specification was correctly formalized. comment ↗
- Experience: A commenter noted that using LLMs to generate Lean code can be problematic because the models often produce excessive, unnecessary code even for simple statements. comment ↗
Summarized opinions from the Hacker News thread, not verified facts. Read the full discussion ↗