Reasoning Models in Production: Claude Extended Thinking vs OpenAI o3 vs Gemini 2.5 Thinking (2026)

Updated: September 7, 2026

Reasoning models are LLMs that spend extra compute at inference time generating a hidden chain-of-thought before producing a final answer, trading latency and tokens for measurably higher accuracy on math, code, and multi-step logic. In production as of 2026, the three families to know are Claude Extended Thinking (Opus 5, Sonnet 5), OpenAI's o-series (o3, o3-pro, o4-mini), and Gemini 2.5 Thinking (Pro and Flash). I've shipped all three behind function-calling pipelines, and the differences that matter are rarely the benchmark scores. They're the pricing of thinking tokens, whether tool calls interleave with reasoning, and whether you can cap the thinking budget.

• Reasoning models generate hidden thinking tokens that you still pay for, typically 3–30x the token count of a normal completion for the same prompt.
• Only use a reasoning model when the failure mode of a non-reasoning call is measurably worse. For most CRUD-shaped LLM work, GPT-4.1, Sonnet 5, or Gemini 2.5 Flash (no thinking) win on latency and cost.
• Claude Extended Thinking is the only family that exposes an integer budget_tokens parameter and streams thinking blocks the developer can log for evals.
• OpenAI's o-series returns opaque reasoning summaries and prices reasoning tokens at the same rate as output tokens. A single o3 call can cost 10–40x a GPT-4.1 call for the same problem.
• Gemini 2.5 Thinking exposes a thinkingBudget parameter with a "dynamic" auto mode and is currently the cheapest reasoning family per thinking token.
• Tool use with reasoning is where vendors diverge most. Claude supports interleaved thinking between tool calls, o-series requires end-to-end reasoning per turn, and Gemini 2.5 supports tool use inside the thinking loop with structured output.

What are reasoning models and how do they work?

A reasoning model is a large language model post-trained (usually with reinforcement learning on verifiable rewards) to emit a long internal chain-of-thought before its user-visible answer. The thinking span is sampled with the same decoding stack as normal generation, but it lives inside a separate token stream that the model conditions on before writing the reply. This test-time compute technique (sometimes called "inference-time reasoning" or "deliberate decoding") moves accuracy gains from bigger pretraining runs to more thinking per query, and it's the single biggest change in LLM productization since function calling landed.

The trade isn't free. Every thinking token is billed. A reasoning-heavy prompt that a normal model finishes in 400 tokens might produce 12,000 hidden thinking tokens plus a 500-token answer on o3. Latency scales the same way. P50 for a hard math prompt on Claude Opus 5 with a 16K thinking budget is typically 30–90 seconds; the same prompt on GPT-4.1 (no reasoning) returns in 3–6 seconds and gets the wrong answer roughly 40% of the time in my regression sets. That last sentence is the whole reason reasoning models exist. On the problem class where they help, they don't just help a little.

Reasoning gains are concentrated on tasks with verifiable structure: math (AIME, MATH), code generation with hidden tests (SWE-Bench Verified, LiveCodeBench), scientific QA (GPQA Diamond), long-horizon planning, and multi-step tool trajectories. On open-ended writing, summarization, or classification, reasoning models often score within noise of their base counterparts, and cost 10x more. If you can't articulate the failure mode a reasoning model is meant to eliminate, don't turn it on.

Claude, OpenAI, and Gemini reasoning models compared

Here's the head-to-head on the axes I actually check when picking a model for a new workflow. Prices are per million tokens as of Q3 2026 and are directional. Always check the vendor pricing pages before wiring in a rate card.

Feature	Claude Extended Thinking (Opus 5)	OpenAI o3 / o4-mini	Gemini 2.5 Thinking (Pro / Flash)
Thinking budget control	Explicit budget_tokens (int)	Coarse reasoning_effort: low/medium/high	Explicit thinkingBudget + dynamic
Thinking token visibility	Streamed thinking blocks (developer-visible)	Summaries only (redacted)	Thought summaries + counts
Interleaved tool calls	Yes (thinking between tool turns)	End-to-end reasoning per turn	Yes, with function calling
Structured outputs during reasoning	Yes (tool-based JSON)	Yes (o3 supports strict JSON)	Yes (responseSchema)
Approx. input price (per 1M)	~$15 (Opus 5) / ~$3 (Sonnet 5)	~$10 (o3) / ~$1.10 (o4-mini)	~$1.25 (Pro) / ~$0.15 (Flash)
Approx. output price (per 1M, incl. thinking)	~$75 / ~$15	~$40 / ~$4.40	~$10 / ~$0.60
Prompt caching for thinking chains	Yes (5m + 1h TTL)	Yes (auto)	Yes (implicit + explicit)
Best at	Long multi-tool trajectories, code refactors, agent runs	Hard math, olympiad-style problems, SWE-Bench	Long-context reasoning, cost-sensitive high-volume

Two patterns pop out. First, price. Gemini 2.5 Flash with thinking is often an order of magnitude cheaper per solved problem than any competitor once you factor in reasoning tokens, and its accuracy on GPQA and MATH is inside 5–10% of Opus 5 for a fraction of the spend. Second, developer control. Only Claude and Gemini let you cap thinking tokens with an integer; OpenAI gives you three effort levels, which is fine for prototypes but painful once you're optimizing p99 latency.

When should you use reasoning models vs standard LLMs?

Reach for a reasoning model when the answer is verifiable and the cost of a wrong answer exceeds the cost of thinking tokens by at least an order of magnitude. Honestly, that covers a narrower slice of production traffic than most people assume. Concretely: SQL generation against a production database, code refactors that will be executed, financial calculations, legal document analysis with citation requirements, and multi-turn agent trajectories with 5+ tool calls all pay for themselves. Chat, summarization, classification, and single-turn extraction generally don't.

My default is a two-tier router. The cheap tier is a non-reasoning model (GPT-4.1, Sonnet 5, or Gemini 2.5 Flash with thinkingBudget: 0) that handles the request. A validator, often a smaller LLM-as-judge or a rule-based check, flags responses below a confidence threshold, and only those get escalated to a reasoning model. In my last three deployments this routing pattern moved 78–92% of traffic to the cheap tier while lifting overall accuracy 8–15 percentage points. For more on cost-aware routing patterns, see the deeper walkthrough in LLM cost optimization with semantic caching and model routing.

Claude Extended Thinking in production

Claude's implementation is the most developer-friendly of the three. You enable it by adding a thinking block with a budget_tokens integer to the request. The API then streams back thinking content blocks before the assistant's text blocks, and you can log both. The thinking blocks are signed. If you pass them back on the next turn (required for tool-use continuations), Anthropic verifies the signature so you can't forge or edit them.

from anthropic import Anthropic

client = Anthropic()

resp = client.messages.create(
    model="claude-opus-5",
    max_tokens=16000,
    thinking={"type": "enabled", "budget_tokens": 8000},
    messages=[
        {"role": "user", "content": "Prove that no odd perfect number less than 10^300 exists, or explain the current state of the problem."}
    ],
)

for block in resp.content:
    if block.type == "thinking":
        # Store for audit / evals; never show raw thinking to end users
        log_thinking(block.thinking, signature=block.signature)
    elif block.type == "text":
        return_to_user(block.text)

Two behaviors matter in production. First, budget_tokens is a soft cap. The model may finish thinking early, but if it hits the budget it stops and produces an answer with whatever it has. Second, if you use tools, you must pass thinking blocks unchanged in the next assistant turn or the API rejects the request. I hit this exact bug shipping a support-triage agent last quarter, and it's a common source of "thinking blocks required" errors in agent loops. It's documented in the Anthropic Extended Thinking guide.

Interleaved thinking, enabled with the interleaved-thinking-2025-05-14 beta header, lets Claude think between tool calls in the same request. For long agent runs this changes everything: the model reflects on tool output before deciding the next action, and my eval scores on the SWE-Bench Verified subset improved 11 points with interleaved thinking on the same budget compared to end-of-turn thinking only.

OpenAI o3 and o4-mini in production

The o-series is the highest-scoring family on hard reasoning benchmarks (o3 sets or matches SOTA on ARC-AGI, GPQA Diamond, and Codeforces), but it's also the most opaque. You don't see the raw chain-of-thought. You get a reasoning field with an optional summary, and you get billed for the underlying thinking tokens at output-token rates. On o3 with reasoning_effort: "high", I routinely see 8,000–30,000 thinking tokens per response.

from openai import OpenAI

client = OpenAI()

resp = client.responses.create(
    model="o3",
    reasoning={"effort": "high", "summary": "auto"},
    input=[
        {"role": "user", "content": "Refactor this Django view to use select_related and prefetch_related correctly. [snip]"}
    ],
)

# Reasoning tokens are billed but not fully visible
usage = resp.usage
print(f"reasoning_tokens={usage.output_tokens_details.reasoning_tokens}")
print(f"answer_tokens={usage.output_tokens - usage.output_tokens_details.reasoning_tokens}")
print(resp.output_text)

Two footguns. First, reasoning_effort: "high" can blow your max_output_tokens budget before the model gets to the visible answer, returning an empty response. Always set max_output_tokens to at least 2–3x what you expect the final answer to need. Second, the o-series doesn't support the temperature parameter and rejects sampling parameters like top_p. The reasoning stack is deterministic-ish and vendor-controlled. See the OpenAI reasoning models guide for the current parameter matrix.

o4-mini is the pragmatic choice for most production reasoning workloads. It's roughly 10% of the price of o3, hits within a few points on most benchmarks, and streams meaningfully faster. If you're shipping reasoning to end users with latency budgets under 15 seconds, o4-mini or Gemini 2.5 Flash Thinking are usually your only options in the OpenAI/Google ecosystems.

Gemini 2.5 Thinking in production

Gemini 2.5 Pro and Flash both support thinking, controlled through a thinkingConfig block with a thinkingBudget integer. Setting thinkingBudget: -1 hands control to the model (dynamic budget), 0 disables thinking entirely (Flash only, useful for splitting traffic), and any positive integer is a hard cap. Gemini also exposes includeThoughts: true which returns thought summaries in the response. Not the raw chain, but more than OpenAI provides.

from google import genai
from google.genai import types

client = genai.Client()

resp = client.models.generate_content(
    model="gemini-2.5-pro",
    contents="Design a database schema for a multi-tenant SaaS with row-level security in Postgres. Explain trade-offs.",
    config=types.GenerateContentConfig(
        thinking_config=types.ThinkingConfig(
            thinking_budget=4096,
            include_thoughts=True,
        ),
    ),
)

for part in resp.candidates[0].content.parts:
    if part.thought:
        log_thought_summary(part.text)
    else:
        return_to_user(part.text)

Gemini's biggest structural advantage is context length. Both 2.5 Pro and Flash support 1M+ token contexts, and reasoning quality holds up well past 100K tokens where OpenAI and Anthropic reasoning models degrade more visibly. For workloads like whole-repo code review, long legal contract analysis, or multi-document synthesis, Gemini 2.5 Pro Thinking is often the only viable option. The Gemini thinking documentation covers the parameter reference and streaming behavior.

The downside: tool-calling ergonomics still lag Claude. Function calling works, but multi-tool trajectories are more likely to loop or return malformed calls than on Claude 5. If your workflow is thinking-heavy and tool-light, Gemini is fantastic. If it's tool-heavy with occasional deep thinking, I lean Claude.

How reasoning models handle function calling and tool use

Function calling is where the vendors most obviously differ, and this is where I have the strongest opinions after enough late-night incident reviews. All three families support tool use with reasoning, but the loops feel completely different.

Claude: interleaved thinking between tool calls

Claude with interleaved thinking runs a fresh thinking span before each tool call and again after tool results return, all inside a single request. The model reflects on partial evidence and adjusts. This mirrors how a human debugs. For agent trajectories with 4+ tools, this is objectively better. The same trajectory that fails without interleaved thinking often succeeds with it, and I haven't seen a case where interleaving hurt.

OpenAI o-series: one big think, then execute

o3 and o4-mini think end-to-end before emitting the first tool call. If the tool returns something unexpected, the model gets another reasoning span on the next turn, but there's no thinking during a turn. This works well when the tool schema is tight and results are predictable, and it fails ungracefully when the model has to react to surprising tool output.

Gemini 2.5: thinking inside the tool loop

Gemini 2.5 will think inside a tool-calling loop when configured with function declarations, similar to Claude's interleaved mode. The behavior isn't as consistent as Claude's (sometimes the model burns thinking budget on the first turn and short-thinks subsequent turns), but it's a reasonable compromise between o-series rigidity and Claude's design.

For a deeper look at how to structure the tool schemas themselves so any of these models can call them reliably, I wrote a separate piece on LLM tool schema design and JSON patterns for reliable function calling. The short version: schema-first design pays back on reasoning models even more than on standard ones, because reasoning models are more likely to notice and refuse malformed schemas.

Cost and latency: the real production numbers

The published benchmarks and the pricing pages together don't tell you what a reasoning model actually costs in your workflow. Here are numbers from a recent internal benchmark on a 100-prompt SQL generation set with 4-tool trajectories and a target of >95% executable, correct queries:

• GPT-4.1 (no reasoning): 87% pass rate, p50 latency 4.2s, ~$0.008 per solved query.
• Claude Sonnet 5 (no thinking): 89% pass rate, p50 latency 5.1s, ~$0.011 per solved query.
• Gemini 2.5 Flash Thinking (dynamic): 94% pass rate, p50 latency 8.3s, ~$0.006 per solved query.
• o4-mini (medium effort): 96% pass rate, p50 latency 11.4s, ~$0.028 per solved query.
• Claude Opus 5 Extended Thinking (8K budget): 97% pass rate, p50 latency 22.9s, ~$0.14 per solved query.
• o3 (high effort): 97% pass rate, p50 latency 41.7s, ~$0.31 per solved query.

The reasoning models hit the accuracy target. The non-reasoning models don't. But at the top of the table, o3 costs roughly 50x GPT-4.1 for a 10-point accuracy lift, while Gemini 2.5 Flash Thinking costs less than GPT-4.1 with a 7-point accuracy lift. Model selection is a Pareto problem and reasoning doesn't always win the Pareto frontier — sometimes it just moves it.

How to evaluate reasoning models before you ship

My rule: no reasoning model touches production traffic without a task-specific eval suite that runs on every model version bump. Reasoning models change behavior between versions more than base models do, because the RL post-training moves substantially between releases. Here's the eval loop I run:

1. Freeze a golden set of 50–500 real prompts pulled from production logs, with human-labeled correct answers and any critical constraints (must call tool X, must return schema Y).
2. Score on three axes: final-answer correctness, tool-call trajectory correctness (did it call the right tools in a valid order), and cost per solved problem. Optionally add a latency SLO check.
3. Run reasoning-off baseline first. If your non-reasoning model hits your accuracy bar, stop. Reasoning won't help and it will cost more.
4. Sweep budget_tokens / reasoning_effort / thinkingBudget. Plot cost vs accuracy. There's usually a knee where an extra 2000 thinking tokens buy 0.5% accuracy. Stop at the knee.
5. Log everything. For Claude, log thinking blocks. For OpenAI, log reasoning summaries and token counts. For Gemini, log thought summaries. When quality regresses, these are the first thing you look at.

For the framework and code around building this loop, my article on building automated LLM evaluation pipelines covers the mechanics end-to-end, and it applies almost unchanged to reasoning models. You just track thinking tokens as a distinct usage metric alongside input and output tokens. Observability adds one more layer worth reading: LLM observability with tracing and monitoring covers how to instrument reasoning traces with OpenTelemetry so they show up in your existing APM.

Frequently Asked Questions

Is Claude Extended Thinking better than OpenAI o3?

Neither is universally better. o3 tops most raw reasoning benchmarks (AIME, GPQA, Codeforces) at the cost of high latency and price. Claude Opus 5 with Extended Thinking wins on agent trajectories, long tool loops, and code refactors, and gives you developer-visible thinking blocks. Pick per workload with a task-specific eval, not a generic benchmark.

How much do reasoning tokens cost compared to normal output tokens?

Reasoning tokens are billed at the standard output-token rate in all three families as of 2026. Because a reasoning response often contains 5–30x more thinking tokens than final-answer tokens, real per-request cost typically ends up 10–40x higher than the same prompt to a non-reasoning model.

Can you cap how many thinking tokens a reasoning model uses?

Claude exposes budget_tokens and Gemini exposes thinkingBudget as integer caps. OpenAI only offers three coarse reasoning_effort levels (low, medium, high). If you need tight latency and cost control, Claude and Gemini give you more knobs.

Do reasoning models work with function calling and tools?

Yes, all three families support tool use with reasoning. Claude offers interleaved thinking (fresh reasoning between each tool call), Gemini supports reasoning inside tool loops, and OpenAI's o-series performs one reasoning span per turn. For long, multi-tool agent trajectories, interleaved thinking on Claude consistently produces the best results in my evals.

When should you avoid using a reasoning model?

Skip reasoning for classification, summarization, chat, entity extraction, and any task where a non-reasoning model already hits your accuracy target. Reasoning models add 5–40x cost and 3–10x latency without measurable accuracy gains on tasks that lack verifiable structure. Always run a non-reasoning baseline first.