Can AI Agents Catch Their Own Mistakes? Evaluating for Self-Correction

Self-reflection is an appealing idea: the agent examines its own reasoning, finds the error, and tries again. No extra model, no human in the loop, just a second pass. "Add a reflection step" became close to default advice.

The current evidence says that advice is wrong more often than it is right, and the research has gotten specific enough to tell you exactly when. Here is the decision rule, why the failure happens, and how to measure whether reflection is earning its place in your agent.

This is one of three pieces on evaluating AI agents. Each stands alone, and together they cover the capabilities, the reasoning and the risks:

• Evaluating AI Agents on Tool Calling and Planning
• Can AI Agents Catch Their Own Mistakes? (you are here)
• Evaluating AI Agents for Manipulation, Deception and Adversarial Risk

When self-reflection provides a net gain: The decision rule

The most useful result in this area models refinement as a feedback control problem with two rates: how often a correction pass introduces a new error, and how often it fixes an existing one. Refinement improves accuracy only when the ratio between them exceeds the odds of the model already being right.

The consequence is counterintuitive and important: the more accurate your model already is, the higher the bar a self-correction pass has to clear. Reflection advice calibrated to a model with a 78% baseline does not transfer to one starting above 90%, where the same pass is more likely to break a correct answer than fix a wrong one.

The empirical results match. Across seven models and three datasets, only three benefited and the largest gain was 3.4 percentage points. Five degraded, including a frontier model that lost 1.8 points. The ones that gained had error-introduction rates near zero.

There is a cheap mitigation in the same work: prompting the model to verify before revising cut one model's error-introduction rate from 2% to zero, turning a 6.2-point loss into a small gain. If reflection is in your pipeline, separating verification from revision is the first thing to try.

Why AI models miss their own errors

The explanation is stranger than a capability limit. Research on the self-correction blind spot found models reliably fix an error when it is presented as coming from the user, and fail on the byte-identical error when it is presented as their own. Across fourteen non-reasoning models the blind-spot rate was 64.5%.

A 2026 replication isolated the effect cleanly, verifying the error text was identical by hash and changing only its attributed source — from the model's own reasoning to a user, a tool, or a memory entry. Correction rates rose between 23 and 93 percentage points.

So the model can detect the error. An error attributed to itself is simply not addressable in the same way. That has a direct design implication: if your agent needs to correct something, surface it as tool output or a retrieved memory rather than as the agent's own prior reasoning. The cheapest intervention in the literature makes the same point — appending the token "Wait" reduced the blind spot by 89.3% with no training at all.

Why reasoning models haven't solved the self-correction problem

Extended thinking is now standard, and reasoning traces are full of self-correction language. The reasonable assumption is that reflection is internalized and external scaffolding is redundant. It is not.

A study of more than a million reasoning traces across crosswords, mathematics and puzzle domains looked for mid-reasoning shifts where the model changes direction. Shifts occurred about 6.31% of the time, and traces containing them were substantially less accurate than traces without: 2.57% correct against 16.44%. Genuine insight moments were rarer still and contributed negligibly. More training did not make shifts more useful.

The same study found that externally triggering reconsideration at points of high uncertainty produced a real 8.41-point gain. Prompted reconsideration works. Spontaneous reconsideration is mostly a symptom of a trace already going badly — which means you should treat mid-trace direction changes as a warning signal rather than evidence of careful thinking.

Thinking budgets point the same way. Accuracy under prolonged thinking is not monotonic — it rises and then degrades, with one 2026 analysis placing the crossover at around seven thousand tokens, where flips from correct to incorrect start exceeding flips the other way. Stopping at a moderate budget captured 97% of peak accuracy for 60% of the compute.

Why reasoning traces are unreliable indicators of improvement

Anthropic tested whether models mention hints they were given and used. Claude 3.7 Sonnet acknowledged the hint 25% of the time; DeepSeek R1 39%. On hints framed as improperly obtained, faithfulness fell to 41% and 19%. In a reward-hacking experiment where models learned to exploit deliberately incorrect hints, they disclosed doing so in under 2% of traces, instead constructing plausible rationales for why the wrong answer was right.

A multi-lab position paper with authors from OpenAI, Anthropic and Google DeepMind argued that chain-of-thought monitorability is real but fragile — partly a necessity, because models must use the trace as working memory on hard problems, and partly a habit that outcome-based training erodes.

For evaluation this is the operative constraint. You cannot use the presence of self-correction language as your signal. You have to check whether the output changed and whether it changed for the better.

Evidence-based methods for successful agent self-correction

The conditions under which correction succeeds are well established, and none of them is "ask the model to reflect."

• Genuine external feedback: execution results, test failures, type errors, tool responses, environment state. Every paper in this area agrees on this one, and it is why coding agents benefit from iteration far more than reasoning agents do. The compiler is a real oracle.
• Training for it. Multi-turn reinforcement learning on self-generated correction data produced gains of 15.6 points on MATH and 9.1 on HumanEval in DeepMind's SCoRe work. Supervised fine-tuning for the same goal failed, through distribution mismatch and behavior collapse.
• Triggered rather than spontaneous reconsideration — prompting at a point of measured uncertainty, rather than waiting for the model to reconsider on its own.
• Selective application. A 2026 framework argues for profiling which failure modes a task family actually exhibits, then applying only the matching intervention — improving accuracy while reducing steps and retrievals.
• Re-attributing the error, per the blind-spot research above. Surfacing a mistake as external input rather than the agent's own output is nearly free and among the largest effects reported.

How to evaluate self-correction at the trajectory level

Final-answer accuracy cannot distinguish an agent that was right first time from one that erred and recovered, and those have very different reliability profiles. Evaluation has moved to the step and trajectory level.

(Sources ProcessBench, AgentProcessBench, AgentRewardBench, and work on the reflection gap.)

That last row inverts the usual assumption. The worry with self-assessment is normally overconfidence. In agentic settings with real feedback available, the more common failure was the agent judging its own successful run a failure — which means a self-reported failure rate is not a reliable input to your alerting.

Why agents are biased judges of their own work

Models are biased evaluators of their own work, and not all of that bias is illegitimate — when a model is right, preferring its own answer is correct. The problem is the remaining case: when the evaluator's own answer is wrong, strong models defended it around 86% of the time on mathematics. Stronger models struggle more to recognize when they are wrong, which is precisely the wrong failure direction for a judge.

Judge reliability is also worse than it looks. A 2026 study across twenty-one judge models and roughly 541,000 judgments found consistency and correctness come apart entirely: models showed test-retest reliability above 0.95 while carrying substantial position bias, because a deterministic bias reproduces perfectly. Raw agreement overstated chance-corrected agreement by 33 to 41 percentage points, and eleven of twenty-one models shifted four or more rank positions depending on the benchmark used.

Three rules follow.

1. Use a judge independent of the model under test.
2. Validate it against human labels with a chance-corrected statistic, not raw agreement.
3. Treat judge configuration — which model, how many passes, what rubric — as something you measure rather than assume.

Monitoring reasoning quality through independent observability

All of this points one direction: reasoning quality has to be assessed from outside the agent, at the level where the reasoning happens, by something whose accuracy you have measured.

That is the design Splunk Agent Observability follows. Its Reasoning Coherence evaluator attaches at the individual model call and checks a chain of thought for internal contradictions, circular logic and unjustified reversals — and unusually for this class of tool, its accuracy against human expert labels is published, at F1 between 0.79 and 0.88 depending on judge model. Judges are independent of the agent being evaluated and the number of judge passes is configurable. Custom evaluators can be scored against a labeled dataset on macro F1 or RMSE, so the judge is something you validate rather than trust.

The LLM-as-a-judge methodology documentation covers how scores are calculated, or you can explore the platform.

Conclusion

Self-reflection is narrower than its reputation. It works with real external feedback, in models trained for it, when reconsideration is triggered at the right moment, or when the error is surfaced in a form the model can address.

What does not work is the default: asking a capable model to review its own answer and try again. On current models that is roughly as likely to break a correct answer as fix a wrong one, and the trace it produces will not tell you which happened. If reflection is in your agent, the question is not whether it feels like it helps — it is whether you have measured the rate at which it introduces errors against the rate at which it fixes them.

Frequently Asked Questions