Definition: Chain-of-thought (CoT) is a prompting technique that asks a large language model to produce its reasoning as an explicit sequence of intermediate steps before giving a final answer. First formalized by Jason Wei and collaborators at Google Research in the 2022 paper "Chain-of-Thought Prompting Elicits Reasoning in Large Language Models", CoT became the foundation for every modern reasoning technique. ReAct, tree-of-thought, self-consistency, reflection, and the explicit "thinking" modes in today's frontier models from Anthropic, OpenAI, and Google all trace back to it.
TL;DR: Chain-of-thought prompting tells a model to show its work step by step instead of jumping to an answer. In the original 2022 study, the same model's accuracy on math word problems rose from roughly 18% to 57% just from this prompt change. Every Taskade AI agent runs on a CoT-native model, so reasoning happens automatically. Build an app that uses it free →
The core insight is counterintuitive. A language model is not a calculator. It does not compute in a hidden scratchpad. Its "reasoning" lives in the tokens it generates. If you ask for only the answer, the model has no room to think. If you ask for the steps, the steps become the thinking. You already do a version of this yourself. When a problem gets hard, you stop trying to answer in one breath and start writing the steps down.
Why Chain-of-Thought Transformed AI
Before CoT, scaling a model meant better recall and smoother text, not better reasoning. Early large models could write a sonnet but failed grade-school arithmetic like 43 × 67. The Wei et al. paper showed that when the same model was prompted to show its work, accuracy on math word problems jumped from roughly 18 percent to 57 percent. Nothing about the model changed. Only the prompt.
This discovery rewired AI research. It established two principles that dominate 2026:
- Reasoning is compute at inference time. You trade tokens for thought. More tokens spent on intermediate steps means more capable output, up to a point.
- Emergent abilities appear with scale. CoT barely helps small models (below ~60 billion parameters). It helps frontier models dramatically. Reasoning is an emergent capability that shows up only when the substrate is large enough.
Every "thinking" model shipping today, including the reasoning lines from Anthropic, OpenAI, DeepSeek, and Google, is a direct descendant of chain-of-thought prompting. The technique is now baked into the model through reinforcement-learning post-training, so the user no longer has to ask for it.
CoT vs Direct Answering
Direct answering gives a model one forward pass to produce a result. Chain-of-thought lets it spend as many passes as the problem needs by writing intermediate steps. On multi-step questions the difference is large: the original study moved math word-problem accuracy from roughly 18% to 57% with no change except the prompt.
| Direct answering | Chain-of-thought | |
|---|---|---|
| Compute per question | One forward pass | Grows with reasoning tokens |
| Best for | Lookups, short facts, classification | Math, logic, multi-step planning |
| Errors | Skips steps, guesses | Mistakes are visible in the steps |
| Cost | Lowest | Higher (more tokens) |
| Trust | Answer only | Reasoning you can check |
The model has no hidden scratchpad. Its reasoning lives in the tokens it generates. Direct answering gives it one forward pass of compute. CoT gives it as many as it needs.
How Chain-of-Thought Works
The mechanism is simple enough to fit in a sentence: ask the model to reason step by step, and include one or more worked examples in the prompt.
Zero-shot CoT (Kojima et al., 2022). Add "Let's think step by step" to the end of the prompt. No examples needed. This is the cheapest form and the one most chat apps default to.
Few-shot CoT (Wei et al., 2022). Include one to eight demonstration problems in the prompt, each with a worked step-by-step solution. The model pattern-matches the demonstration style when answering the new question.
Self-consistency (Wang et al., 2022). Generate many CoT responses at a higher temperature and take the majority-vote answer. Costs more tokens but boosts accuracy on hard problems.
Automatic CoT (Auto-CoT). The model clusters questions, generates exemplars for each cluster, and constructs the CoT prompt automatically. Useful when you do not have hand-written examples.
Tree-of-thought and graph-of-thought. Instead of one linear chain, the model explores multiple branches, evaluates them, and backtracks. Think chess engine for language. Costs 10 to 100 times more tokens but unlocks tasks single-chain reasoning cannot solve.
The Mechanics: Why Tokens Are Thought
A transformer generates one token at a time, attending to all previous tokens. The model has no memory between calls, only the prompt and the output so far. That means every intermediate step the model produces becomes part of the context for the next token. The chain feeds itself.
Take the question "If Alice has 3 times as many apples as Bob, and together they have 24, how many does Alice have?" A direct-answer model has one forward pass to produce "18." A CoT model walks the chain instead, and each step it writes becomes the ground the next step stands on:
Every intermediate token gives the model additional depth. The math happens in the output stream. You can see the same idea as a token timeline, where each written step is reused as context for what comes next:
PROMPT ─► [ Let Bob = x ] ─► [ Alice = 3x ] ─► [ 4x = 24 ] ─► [ x = 6 ] ─► [ 18 ]
│ │ │ │
└── each step is fed back in as context for the next token ──┘
This is why CoT is strictly more powerful than equivalent-parameter direct answering. The model's effective depth for a hard question grows with the number of reasoning tokens it writes.
CoT vs Agentic Reasoning
CoT is a single-turn pattern. The model reasons and answers, all inside one response. Agentic AI extends CoT across many turns, interleaving reasoning with tool use.
| Pattern | Scope | Example |
|---|---|---|
| Chain-of-thought | One turn, no tools | Solve a math problem |
| ReAct | Multi-turn, reasoning + tools | Answer a question by searching the web |
| Tree-of-thought | Branching one-turn | Solve a game state |
| Agentic RAG | Multi-turn retrieval + reasoning | Research report from many sources |
| Multi-agent | Multi-agent, each using CoT | Manager delegates subtasks to specialists |
The line between CoT and agentic reasoning has blurred since 2024. Frontier models now do internal chain-of-thought before every visible response, and agentic systems like Taskade Genesis use CoT at every step of a multi-turn automation.
Chain-of-Thought in Practice
Three patterns appear in almost every production system:
Hidden CoT. The model thinks step-by-step, but only the final answer is shown to the user. This is how most reasoning models work by default. Reasoning tokens are still billed but not rendered.
Visible CoT. The reasoning steps are shown in the UI, often in a collapsible "thinking" panel. Useful when users need to trust the answer or catch errors. Taskade EVE's build logs follow this pattern during Taskade Genesis app generation. For a plain-English guide to choosing how much reasoning a task needs, see Thinking Modes in Learn Taskade.
Structured CoT. The reasoning is constrained to a specific schema: hypothesis, evidence, counter-evidence, conclusion. Common in enterprise analytics agents where explainability is regulated.
Failure Modes
CoT is not a silver bullet. Known pitfalls:
1. Faithfulness gap. The steps the model shows do not always match the computation it used internally. A model may produce the correct answer despite an incorrect-looking chain, or vice versa. This matters for safety-critical applications.
2. Verbose drift. Without a token budget, CoT can ramble. Set a maximum output length or structure the chain with headers to keep it focused.
3. Confidence inflation. Models that produce long chains sound more certain even when they are wrong. Combine CoT with self-consistency or grounded retrieval to catch this.
4. Emergent-only. CoT does little for small models. If you are running a 7B or 13B local model, expect modest gains. Reserve CoT for frontier-class models.
Chain-of-Thought in Taskade Genesis
Every Taskade AI agent runs on a CoT-native frontier model. You do not have to prompt for reasoning. The model reasons by default before calling tools, responding to users, or executing automations. Taskade routes the right model to each job automatically through its Auto setting, drawing on 15+ frontier models from OpenAI, Anthropic, Google, and open-weight providers.
When Taskade EVE, the meta-agent behind Taskade Genesis, builds a Taskade Genesis app from a prompt, the process is a chain-of-thought at every level:
- Parse the user's intent (CoT reasoning)
- Draft a plan of tasks and components (CoT reasoning)
- Call build tools in sequence, observing each result (CoT + tool use = ReAct)
- Verify the result against the plan (CoT reflection)
- If something is ambiguous, call the Ask Questions tool and wait for the user
The same pattern runs in your automations. The automation triggers, the agent reasons about the incoming payload, the agent calls tools, the agent reasons about the results, and the automation completes. With 34 built-in tools behind each agent, the reasoning chain can search, calculate, and act, not just talk.
Build a Tool That Reasons Step by Step
You do not have to be an engineer to put chain-of-thought to work. Describe the decision you keep making by hand, and Taskade Genesis builds an app where an agent reasons through it for you.
Picture an operations dashboard for your business. Every morning it pulls in the day's new requests, an agent reasons over each one step by step (urgency, owner, next action), and it sorts them into the right lane before anyone logs in. Your team opens one screen and sees decisions already made, with the reasoning shown so they can trust or override it. Nothing to wire up, no model to pick. The right one is chosen automatically.
You already do a version of this in your head or a spreadsheet every morning. The app just does the steps for you and keeps doing them on its own. Describe yours and build it free →
Related Concepts
- ReAct Pattern: CoT interleaved with tool calls
- Prompt Engineering: the discipline CoT belongs to
- Prompt Chaining: multi-prompt workflows
- Agentic AI: why CoT is the foundation
- Tool Use: what CoT enables in agents
- Large Language Models: where CoT works and why
- Emergent Behavior: why CoT only appears at scale
- AI Agents in Taskade: agents that reason with CoT by default
- Thinking Modes (Learn Taskade): pick the right reasoning depth per task
Frequently Asked Questions About Chain-of-Thought
What is chain-of-thought prompting?
Chain-of-thought (CoT) prompting asks a large language model to produce its reasoning as an explicit sequence of intermediate steps before giving a final answer. It turns language generation into a visible reasoning process and dramatically improves accuracy on multi-step problems.
Who invented chain-of-thought?
CoT was formalized by Jason Wei and collaborators at Google Research in the 2022 paper "Chain-of-Thought Prompting Elicits Reasoning in Large Language Models." A related technique, zero-shot CoT ("Let's think step by step"), was introduced by Takeshi Kojima and collaborators the same year.
Does chain-of-thought work on every model?
No. CoT is an emergent capability that shows up only in sufficiently large models (roughly 60+ billion parameters). Small models produce reasoning-shaped text without the underlying competence. Frontier models from OpenAI, Anthropic, and Google all use CoT effectively; most small open-source models do not.
What is the difference between CoT and ReAct?
Chain-of-thought is a single-turn reasoning pattern. The model reasons and answers in one response. The ReAct pattern extends CoT across multiple turns by interleaving reasoning with tool calls, so the model can observe the world between steps.
Does chain-of-thought always show how the model really reasoned?
Not always. Research from Anthropic and others in 2025 found that a model's visible reasoning steps do not always match the computation it used internally. The chain can still help accuracy, but treat it as an aid for trust and review, not a guaranteed transcript of the model's internal process.
How do I get a model to use chain-of-thought?
Add "Let's think step by step" to your prompt (zero-shot CoT), or include a few worked examples that show the steps (few-shot CoT). Most reasoning models from OpenAI, Anthropic, and Google now do this on their own. In Taskade you choose how much reasoning a task gets with Thinking Modes.
How do Taskade agents use chain-of-thought?
Every Taskade AI agent runs on a CoT-native frontier model. Taskade's Auto setting picks the right one from 15+ frontier models across OpenAI, Anthropic, Google, and open-weight providers. The model reasons step-by-step before tool calls, during automations, and while building Taskade Genesis apps. You do not prompt for CoT; the models do it automatically.
Further Reading
- What Is Agentic AI?: why CoT is the base layer
- What Are AI Agents?: how CoT becomes autonomy
- Multi-Agent Collaboration: Production Lessons
- The Perceptron: the long arc from weights to reasoning