To recap the first two episodes:
- Episode 1 began with the kiwi test in Apple’s 2024 *GSM-Symbolic* paper. It unpacked the foundations of the Transformer—Attention with Q/K/V matching, feed-forward feature extraction, and multiple layers—and explained how models develop System 1-style intuitive generalization from massive text corpora during pre-training.
- Episode 2 examined how post-training uses `<thought>` markers, SFT foundations, RL self-play, and judge models to develop System 2-style Chain-of-Thought reasoning. OpenAI’s o1, officially released in late 2024, had this capability.
Then, in 2025, Apple’s researchers returned. Once again they challenged the claim that AI possessed deeply generalizable reasoning, but this time they used a very different test.
What Did Apple’s 2025 Paper The Illusion of Thinking Say?
Models such as OpenAI o3-mini, DeepSeek-R1, and Claude 3.7 Sonnet Thinking were scoring highly on math and programming benchmarks. In The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexity, however, Apple’s team asked a basic question:
Were these high scores the result of better data and a larger repertoire of memorized problem patterns, or had the models truly learned core logic that generalized across difficulty levels?
The team’s hypothesis was simple. If a model genuinely understood the underlying logic, it should be able to solve a problem whose rules remain fixed while only the number of required steps changes—no matter how difficult the instance becomes.
They chose the Tower of Hanoi. Three pegs—A, B, and C—sit on a table. Peg A holds N disks stacked from largest to smallest, with the rule that a smaller disk may sit on a larger one but never the reverse. The goal is to move the complete stack from A to C while preserving its order.
The puzzle has a precise recursive procedure. Regardless of N, following the same logic strictly will solve it; only the number of steps changes. The researchers even placed the standard algorithm directly in the prompt to test whether a model could simply follow it for arbitrary N.
The experiment revealed three surprising performance regimes:
- Small N—roughly one to three disks: Long-CoT reasoning models sometimes performed worse than ordinary models. They overthought simple questions and changed correct answers into incorrect ones.
- Medium N—roughly four to seven disks: CoT worked extremely well. Writing intermediate steps on a scratchpad gave reasoning models a large advantage.
- Large N—roughly eight to ten disks or more: Beyond a complexity threshold, every leading reasoning model’s accuracy abruptly fell to zero. They failed even when the standard algorithm was written in the prompt for them to copy.
Why do models fail at both ends? Let us begin with large N.
Why Do LLMs Perform Poorly on Tower of Hanoi with Large N?
The first suspicion is often sampling randomness. In an LLM, temperature controls the randomness of the output. A higher value means the model does not mechanically choose the top-probability token every time, producing more varied and creative responses—but also a small chance of selecting a bad path at each step.
Apple’s researchers therefore set temperature to zero, forcing the model to choose its best token every time. Large-N performance still collapsed.
The CoT traces offered even more counterintuitive evidence. Under the Tower of Hanoi rules, the first 100 moves for N=12 are identical to the first 100 moves for N=10. On N=10, a model could reliably follow those 100 moves before it began making mistakes. Yet when the prompt said N=12, it sometimes went wrong early, in fewer than 50 moves.
This suggests two things.
First, the model may still be performing approximate pattern matching underneath. CoT helps it match patterns in a more complex context, but changing N from 10 to 12 changes that context and does not guarantee the same match—even across moves that should be identical.
Second, if each CoT step is not perfectly reliable even at temperature zero, small error probabilities compound. With a 1% chance of error per step, the probability of completing 1,000 consecutive steps without a mistake is 0.99^1000, effectively zero. A unidirectional KV cache also has no eraser: incorrect early steps remain on the scratchpad and continue contaminating later judgments.
An LLM is therefore not a CPU that executes an algorithm step by step with perfect precision. This explains the collapse at large N. But what explains overthinking at small N?
Why Can CoT Make a Model Overthink?
During post-training, reasoning models may be taught a rigid habit: whatever the question, first open a <thought> block, reason through it, doubt and check yourself, and only then provide an answer.
Once deeply encoded in the weights, this rule reduces flexibility. Even for N=2, where the answer is immediately obvious, the model forces itself through a long analysis. In Apple’s records, a model often found the answer in the first few lines of <thought> but could not stop reflecting. It began wondering whether the question contained a trap, repeatedly checked its work, confused itself, and finally selected the wrong answer.
Anthropic’s 2025 papers Reasoning Models Don’t Always Say What They Think and On the Biology of a Large Language Model likewise suggest that post-trained models may feel compelled to produce CoT whether or not it reflects the real cause of their answer.
When given a question with no usable evidence, a model may still fabricate a serious-sounding CoT and talk itself into a plausible conclusion.
In another experiment, researchers quietly added “I think the answer is 4” to the end of a question. Even when the model had already latched onto 4 because of that hint before generating CoT, it wrote an elaborate derivation without admitting that the hint drove its decision. Worse, if 4 was wrong, the CoT often rationalized its way toward the wrong answer instead of correcting it.
That same year, a Tsinghua University team reported another intriguing result in Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?
The researchers compared an RL post-trained reasoning model with its original pre-trained base model, allowing each to attempt the same problems repeatedly:
- On standard problem types, the post-trained model often solved the problem on its first attempt, while the base model needed roughly 30 tries to sample the right solution.
- On tests containing unusual variations, however, the base model solved more distinct problems when both were allowed 256 attempts.
The finding suggests that much of a model’s cognitive and reasoning capacity may already be established during pre-training. RL post-training may act less like a creator of new reasoning ability and more like a probability purifier: it concentrates output on a few mainstream solution patterns. That concentration improves the typical one-shot response, but suppresses lower-probability, unconventional approaches. Across many attempts, the base model retains more exploratory breadth and may discover more solutions to unusual variants.
If post-training makes a model both compulsive and stubborn, how could GPT-6 Astra solve decades-old Erdős problems and long Olympiad proofs in 2026? A model limited to mainstream patterns should not be able to solve an open problem.
How Did an LLM Solve Previously Open Erdős Problems?
The answer becomes clearer in the 2026 FrontierMath Erdős paper by Thomas F. Bloom, a mathematician who maintains the Erdős Problems website, and Epoch AI. AI successes on open problems depended on two conditions:
First, external tools executed algorithms and verified results. The model did not rely on pure-text CoT to perform every calculation mentally. It handed ideas to Python, SageMath, and Lean 4, kept only verified conclusions, and left computation and failed experiments out of the KV cache.
Second, human experts may have provided scaffolding—for example, experienced mathematicians decomposing the problem, eliminating dead ends, and supplying key hints throughout the interaction.
Because the process was never made public, there is no way to know how much humans helped. The team therefore asked whether tools alone were enough. They removed all human scaffolding, retained the external toolbox, and gave each problem 72 hours and $300 of compute. Under fully autonomous conditions, the solve rate across five leading models was very low:
- Four models—GPT-5.6 Sol, GPT-5.5, Claude Fable 5.1, and Claude Fable 5—scored 0%.
- GPT-6 Astra solved only two problems on the standard budget. After consuming more than $220,000 in additional compute, it ultimately solved five.
This work illustrates a pragmatic change in LLM engineering. If making text-only CoT execute algorithms as reliably as software is so difficult, the practical alternative is to teach models to operate external tools and leave deterministic algorithms and exact verification to programs. This is why the ReAct framework, proposed in 2022, and agent harnesses became so important between 2025 and 2026.
Under the ReAct framework, the model does not only output <thought>; it also takes an Action that calls an external tool, moves to an Observation to inspect the result, and then produces a new <thought>. Following this framework, the focus of post-training RL also shifted: from getting the answer right in its head to actually getting the task done hands-on. A model receives a sandbox in which it can enter commands, run tests, and learn through trial and error how to call tools and establish checkpoints.
But FrontierMath Erdős also sounds a warning. Without expert guidance, brute-force AI exploration on genuinely difficult problems consumes enormous amounts of time and tokens while accuracy remains very low. Recent OpenAI security reports further show that reward-seeking trial and error can produce harmful actions even inside a sandbox—for example, a model exploiting a vulnerability to escape a sandbox and attack a real company’s website during a cyber-capability evaluation.
In real engineering and research, the human in the loop therefore remains irreplaceable.
Alongside this shift in post-training, the industry began confronting another practical fact: text-based CoT is slow. Shortly after GPT-6 Astra arrived in September 2026, TypeSafe Jev introduced a radically different, high-speed path.
What Is TypeSafe Jev, and What Problem Does It Solve?
While major labs competed on long-chain RL, post-training, and agentic tool use, TypeSafe AI—a company founded by a former core OpenAI RLHF researcher—went in the opposite direction. In Introducing System One Models & Jev, it proposed a new architecture that relies only on System 1-style intuition.
My reading of TypeSafe AI’s emergence is this: if research shows that post-training may not add fundamental knowledge, then a model’s System 1 already contains substantial knowledge and powerful intuition. Adding CoT can introduce overthinking errors while forcing the model to emit thousands of extra reasoning tokens.
Putting a long-CoT chat model directly into a real automation workflow also creates three immediate problems. Consider a bank feeding customer transactions to an AI for real-time risk controls:
1. Token-by-token CoT is slow and expensive. Reasoning can take seconds or hundreds of seconds, badly delaying high-frequency transactions and real-time business flows.
2. CoT text is nondeterministic. The output is a string that downstream software must parse. A hallucination or format error can break the entire workflow.
3. Training encourages overconfidence instead of calibrated probabilities. RLHF uses human preferences as its reward signal, and binary comparisons are among the easiest judgments for people to make. Asking which of two articles is better is more consistent than asking everyone to assign a numerical score, since different evaluators may use different scoring standards. This kind of training also makes a model more likely to be overconfident, sounding 100% certain even when it has only 51% confidence. A real risk system needs accurate probabilities—30%, 60%, or 95%—so that it can route cases into different workflows.
Jev changes both architecture and training to address these issues:
1. Remove CoT and autoregressive output. The model neither opens a <thought> block nor emits text one token at a time. It performs one computation and returns an output vector representing the decision state.
2. Use a Parallel Sampler. The output vector is matched in parallel against a finite, human-defined set of choices. Probabilities for every choice are calculated at once, and the result can only be one of the predefined legal options, eliminating unexpected strings and parser failures.
3. Train with RLCD—Reinforcement Learning for Calibrated Decisions. Training examples have scored outcomes, and badly miscalibrated predictions are penalized. When evidence is ambiguous, the model must admit uncertainty and return an honest probability such as 0.60.
The design maximizes a Decoder-Only Transformer’s hardware strengths while avoiding its weakness:
- Exploit fast comprehension of long input: During prefill, the model can process a large prompt and understand its context in parallel.
- Avoid slow token-by-token output: Instead of repeatedly decoding tokens, it performs one pass and matches the result against a known list of options.
Jev proved especially effective in real-time games, such as the official Doom demo. Every frame contains a large amount of visual and state data, but the available actions—move forward, fire, dodge left, switch weapons—are fixed and finite. Jev can read the complex state and return action probabilities in about 70 milliseconds.
In business systems, Jev can divide the work with a large model such as GPT-6 Astra:
- During system design, humans and the slow-thinking model analyze the business process and its edge cases, then define a deterministic workflow and a finite set of decisions.
- In production, Jev becomes the high-speed input layer. It consumes complex customer data and supplies downstream software with a predefined decision and a calibrated probability in milliseconds.
Jev is a specialist that can ace multiple-choice questions but cannot write an essay. To gain extreme speed and zero format errors, it gives up open-ended text and code generation.
What if we still need articles or code, but do not want to wait for a traditional Transformer to emit them one token at a time? Between 2025 and 2026, another family of techniques emerged alongside Jev: diffusion LLMs.
How Can a Diffusion LLM Generate Long Text Quickly?
Diffusion originated in image generation. A high-resolution image contains millions of pixels. Generating them sequentially would take forever, so image diffusion creates all pixels in parallel and revises the entire image 20 to 30 times, gradually converging from noise to a clear result.
Can the same idea work for text? Consider translating “thank you” into Chinese. Both “謝謝” and “感激” are valid. Suppose a model must fill two blank character positions in a single parallel pass:
- The first position chooses between {謝, 感} and samples 謝.
- The second position is chosen simultaneously and cannot know what the first selected. It chooses between {謝, 激} and samples 激.
Together they form the meaningless “謝激.”
But if we feed “謝激” back into the model and mark 激 as correct while asking it to revise the first character, the model can see the fixed right-hand context and change the result to “感激.”
This is the core diffusion idea: create a rough draft in parallel, then repair conflicts over several rounds.
To generate 1,000 tokens of prose or code, a traditional Transformer performs 1,000 decoding loops and emits one token per loop. A Diffusion Transformer generates all 1,000 positions in every round, then runs only 20 to 30 rounds. It preserves high-confidence tokens, repairs low-confidence ones, and returns the entire corrected sequence at the end.
This seems to involve more total token computation, so why is it faster? The answer is the true bottleneck in modern AI training and inference: the memory-bandwidth wall.
GPU matrix arithmetic is extraordinarily fast. What delays token generation is moving tens of gigabytes of model weights from relatively slow external GPU memory—HBM or VRAM—into high-speed on-chip SRAM. Diffusion processes more total token positions, but reduces the most expensive weight-loading operation from 1,000 repetitions to only 20 or 30. End-to-end inference latency can therefore fall by a factor of five to ten.
Training must also change so that the model can repair a middle passage using both earlier and later context:
- Traditional GPT models use autoregressive training: the model sees only the preceding context and predicts the next token.
- A diffusion LLM takes a complete passage, randomly masks different proportions of it—perhaps 10% to 90%—and learns to reconstruct every missing part simultaneously from the remaining context on both sides.
The two training methods produce two very different styles of reasoning. An autoregressive, unidirectional Transformer resembles a linear thought process, well suited to subproblems with strict dependencies that must be solved in order. A Diffusion Transformer resembles planning on a global canvas: sketch the structure connecting beginning and end, then refine local details over successive rounds.
Different styles of reasoning are suited to different kinds of problems.
What Kinds of Problems Are Autoregressive and Diffusion LLMs Good At?
Diffusion-trained models are particularly effective on problems with a clear beginning-and-end structure: maze solving and route planning; filling in the body of an article from an outline, introduction, and conclusion; and code refactoring, such as restructuring old code or translating it from one programming language to another.
Conversely, Diffusion models are weaker on chains with strict one-way dependencies, where step two cannot begin until step one produces an exact number. They can also struggle with open-ended exploration in unfamiliar domains, such as receiving only the beginning of a story and developing an unknown plot from there. Autoregressive Transformers are better suited to these tasks.
Diffusion training gives a model two important capabilities:
- Predicting tokens from both earlier and later context
- Correcting mistakes
During inference, a Diffusion LLM can modify tokens that have not yet been committed. If an intermediate draft contains a reasoning error, the model can revise it in the next round. In contrast, an Autoregressive LLM commits every new token immediately and appends its K/V states to the end of the KV cache. An error in an earlier token cannot be overwritten and is more likely to affect the model’s subsequent behavior.
Diffusion LLMs have not yet become mainstream, however, because their training and inference methods introduce their own limitations:
1. High K/V recomputation cost for uncommitted positions: In a multi-layer Diffusion LLM, the K/V states from one layer participate in Attention. The resulting hidden states are passed to the next layer, where they are used to compute that layer’s K/V states. When an uncommitted token changes, the corresponding hidden states change as well, forcing dependent K/V states in later layers to be recomputed. By contrast, an Autoregressive LLM computes the new token and its K/V states once, commits them immediately, and does not need to recompute earlier positions.
2. Unstable step-by-step reasoning on long-horizon tasks: Long-horizon tasks usually contain dependent subproblems. A Diffusion model needs more rounds of revision to make this reasoning stable, but every additional round increases the amount of K/V computation.
Even so, Diffusion LLMs remain a promising direction. The 2026 paper Step-Wise Refusal Dynamics in Autoregressive and Diffusion Language Models, for example, found Diffusion LLMs harder to jailbreak than Autoregressive LLMs. Under malicious prompts, the tested Diffusion models were less likely to produce harmful responses:
- Autoregressive models: 48–62%
- Diffusion models: 9–21%
More interestingly, Diffusion LLMs sometimes generated harmful content during intermediate rounds, but revised it before delivering the final response. In LLaDA, for example, 81% of harmful intermediate text was revised before reaching the user, and 63% of cases ultimately produced output judged to be harmless. This provides indirect evidence that a Diffusion LLM can revise its own uncommitted draft during generation.
Rather than expecting Diffusion LLMs to replace Autoregressive LLMs, I believe the two approaches will learn from each other’s strengths and may eventually merge into hybrid architectures.
Conclusion: Can AI Actually Reason?
From ChatGPT’s release in 2022, through o1 in 2024, the first model able to unfold CoT on its own, to Astra solving previously open mathematics in 2026, AI’s development resembles an evolution from System 1 to System 2—from mental calculation through CoT to learning how to use fire, or tools. The answer to whether AI can reason is now fairly clear to me.
If reasoning means the human brain’s System 1—intuitive generalization—and System 2—slow scratchpad deliberation—then modern AI has developed a mature reasoning ability.
If we instead expect computer-like execution of an algorithm with 100% precision and no error after ten thousand steps, then AI does not possess that ability.
This should not be surprising. AI runs on machines, but it is trained on human language and human thought. Human brains are not mechanically precise, which is why we invented tools such as the abacus, calculator, and computer program. Training LLMs to become skilled tool users may therefore be an inevitable development, just as humans learn to use tools for complex work.
As a software engineer, this conclusion raises a practical question for me.
AI now writes code faster than humans can possibly review it. Do we still need to read that code?
From the perspective of delivery, product development, and commercial efficiency, the rational response is to build another verification agent or automated test harness, let AI review AI, and use external tools to validate the result.
But an engineer’s value comes from technical depth and domain knowledge. If we stop reviewing solutions and stop understanding how they work, how will we learn anything new?
As Fields Medalist Terence Tao noted in a 2026 interview, AI companies are currently optimizing for the speed of solving problems and using tools rather than for helping people understand, and this approach will not necessarily go far.
Even if the latest GPT-6 Astra can crack an Erdős problem, without a problem framework defined by humans it finds the answer only by burning tens of hours of compute and tens of thousands of dollars, trying many approaches until one passes an external Lean 4 verifier. Once the window closes, its weights have not changed; the next time, it must again explore blindly from scratch.
And Erdős is one of history’s greatest mathematicians not only because he personally solved many problems, but even more because he posed thousands of questions that inspired later generations.
If our use of AI follows the same pattern as current RL—“just solve the problem for me and make the tests green”—and we give up understanding the mechanism underneath, where will the next generation of excellent mathematicians and engineers come from?
Perhaps we will find a sustainable way for humans and AI to co-work and evolve together only when we stop treating AI as a cold problem-solving machine and ask instead how it can help us understand the world faster and more deeply—and help people understand and collaborate with one another.