LLM vs RAG vs AI Agent vs Agentic AI: The Architecture Guide (2026)

Last verified 13 October 2026. Every cost, latency and failure-rate figure below links to its primary source.

There are four ways to build on a language model, and the entire industry is skipping to the fourth.

This is a guide to the other three — what each one actually does, what it costs, how it breaks, and the procedure for deciding which one your problem needs. The short version: each level buys a capability the level below cannot express, and charges you a failure mode the level below does not have. The right architecture is the lowest one that clears your requirement, not the highest one you can build.

TL;DR: LLM, RAG, AI agent, and agentic AI differ by one variable — who decides how many model calls happen. Nobody, your code, the model, then an orchestrator. Each rung adds capability and a new failure mode, and RAG to agent measured 36× the cost. Most systems should stop at a workflow, the rung everyone skips. Build on the right rung →

The Four Rungs at a Glance

The four architectures are usually presented as a taxonomy — four boxes, four definitions. That framing hides the thing that matters. They are a ladder, and what changes as you climb is control.

The One Variable That Separates Them

Rungs 1, 2 and 2.5 are you writing the control flow. Rung 3 hands control flow to the model. Rung 4 adds an orchestrator that drives one or more models toward a shared objective over shared state — the orchestrator and the shared state are the additions, not the agent count.

That single handoff — control flow moving from your code into the model — is the most consequential architectural decision in this entire space, and it is the one most often made by accident.

What Each Rung Costs

Note the cost shape column. Rungs 1, 2, and most of 2.5 have flat costs: you can put a number on a request before you run it. One workflow pattern is the exception — an evaluator-optimizer loop revises until it passes, so only its cap is known in advance, not its actual count. Rungs 3 and 4 have full distributions, and the tail of that distribution, not the median, is what sets your bill and your p95 latency.

Rung 1: The LLM

An LLM takes a prompt and generates a response from its learned parameters, predicting tokens one at a time. That is the entire mechanism. There is no lookup, no memory, and no verification.

What a Bare LLM Is Genuinely Good At

It is easy to dismiss rung 1 once you know about the others. Do not. A single well-constructed call is the highest-reliability, lowest-latency, cheapest option available, and an enormous share of real work fits inside it:

• Transformation — rewrite this in a different register, convert this format to that one
• Summarization — condense text that is already in the prompt
• Classification and extraction — route this ticket, pull the fields out of this email
• Drafting — produce a first version a human will edit
• Reasoning over provided material — the facts are in the prompt already

The common thread: everything the model needs is already in front of it. No lookup required.

Where a Bare LLM Fails

The first three are all the same failure wearing different clothes: the model has no way to distinguish what it knows from what it is generating. That is precisely the gap rung 2 exists to close.

When to Stop at Rung 1

Stop here when the task is a transformation of material you already have. If you are tempted to climb because output quality is poor, check the prompt first — a bad rung-1 result is far more often a prompt problem than an architecture problem, and climbing a rung to fix it means you now own two problems.

Rung 2: RAG (Retrieval-Augmented Generation)

In RAG the query goes to a retriever before it reaches the model. The retriever searches an indexed knowledge base, pulls the relevant chunks, and passes them to the LLM alongside the original query. The model generates a grounded response.

Read the last box carefully. Grounded is not the same as correct. Retrieval changes what the model sees; it does not change what the model does with what it sees. This distinction is the single most expensive misunderstanding in production RAG systems, and it deserves its own section.

The RAG Failure Taxonomy

Almost every RAG failure is a retrieval failure, not a generation failure. Teams debug the model for weeks when the problem was three layers earlier.

Six distinct failure points, and only one of them lives in the model.

The Semantic Gap Is Worse Than People Expect

The third check deserves elaboration because it is counter-intuitive. Embedding search assumes a question sits near its answer in vector space. Often it does not.

"What is our refund window?" and "Customers may return items within 30 days of delivery for a full refund" share almost no vocabulary. A pure dense-vector search can miss it entirely, while a keyword search for "refund" finds it instantly.

This is why hybrid retrieval — dense vectors plus lexical search, fused — outperforms either alone on most real corpora. If your RAG system mysteriously misses obvious answers, this is the first thing to test.

More Retrieval Can Make Things Worse

There is a strong instinct to raise top-k when RAG underperforms. Research on long-context behavior has repeatedly found the opposite effect: adding distractor passages degrades accuracy even when the correct passage is still present, and models attend unevenly across a long context. Chroma's context rot research, which tested 18 models across 194,480 calls, also reported a genuinely surprising result — a coherent, well-structured haystack can be harder for a model to search than the same content shuffled.

The practical rule: retrieve precisely, not abundantly. A reranker that returns three excellent chunks beats a retriever that returns twenty adequate ones.

What RAG Is Genuinely Good At

• Question answering over a corpus you own
• Support and documentation lookup, where citations matter
• Policy and compliance questions where the source must be quotable
• Anything where the answer exists in text somewhere and needs finding

When to Stop at Rung 2

Stop here when the job is to answer rather than to act, and when one retrieval round can supply the facts. If answering requires several dependent lookups — where what you search for second depends on what you found first — you have hit the ceiling of plain RAG, and the next section is where you go.

Rung 2.5: The Workflow Everyone Skips

Between "one call with retrieval" and "let the model decide" sits the rung that most production systems should actually occupy, and it usually goes unnamed.

Anthropic draws the line precisely in Building effective agents:

Workflows are systems where LLMs and tools are orchestrated through predefined code paths.
Agents are systems where LLMs dynamically direct their own processes and tool usage.

That is the whole distinction, and it is about who wrote the control flow, not about how sophisticated the system looks. Their guidance is equally direct: find the simplest solution possible, and only increase complexity when it demonstrably improves outcomes.

The Common Workflow Patterns

Why the Workflow Rung Matters So Much

A well-built workflow beats a badly-built agent on every axis that matters in production: latency, cost, reliability, testability, and your ability to understand what happened at 2am.

The reason this rung gets skipped is not technical. "We built an agentic system" is a better sentence than "we wrote a routing function." But the routing function ships, runs for a tenth of the cost, and can be unit tested.

The Data Says Production Already Agrees With This

This is not a contrarian opinion. It is what shipped systems actually look like.

An analysis of real production deployments found that 80% (16 of 20) used predefined structured workflows rather than open-ended autonomous planning, prioritizing controllability and human oversight over autonomy (arXiv:2512.04123). That is a survey of case studies rather than a random industry sample, but the direction is unambiguous.

Anthropic's own January 2026 guidance is blunter still:

"Today, multi-agent systems are often applied in situations where a single agent would perform better."

And in the same post: "Teams have invested months building elaborate multi-agent architectures only to discover that improved prompting on a single agent achieved equivalent results."

If you can write the steps down before you run anything, you do not have an agent problem. You have a workflow, and you should build one.

A Measured Case Where Adding a Tier Made Things Worse

A controlled experiment appended naive RAG onto an expert-designed workflow for MATLAB-to-HDL code translation. Success rate dropped from 33.5% to 19.5% at 30B model scale — attributed to context clutter, architectural mismatch in the retrieved examples, and truncation of precise compiler errors (arXiv:2512.14762).

Adding retrieval to a working system cut its success rate by nearly half.

The same paper carries a second finding that complicates every rule in this article, and it is worth stating honestly: the right tier depends on the model. For 8B–30B models, an agentic approach with conditional retrieval beat the fixed workflow by 20+ percentage points. At 235B scale, the agentic advantage shrank and naive RAG became comparable.

So "use the lowest rung that works" is the right default, but which rung works moves as models improve. Re-test your architecture decision when you change models — it is not a permanent answer.

Rung 3: The AI Agent

An agent has a goal and keeps track of the current task state. It plans what to do next, calls a tool, gets a result, observes it, and repeats until the goal is met. The feedback loop is the point: the agent adjusts its actions based on what actually happened.

This loop descends from ReAct (Yao et al., 2022), which interleaved reasoning traces with actions and showed that a model that could both think and act outperformed one that only did either. Every modern agent framework is a descendant of that pattern.

The Agent's Defining Risk: Errors Compound

This is the most important arithmetic in agent engineering, and it is simple enough to do in your head.

If each step succeeds independently with probability p, then a task needing n sequential steps succeeds with probability p^n.

Read the 95% row again. A step reliability that would be excellent for a single API call produces a 36% completion rate over twenty steps. This one table explains nearly every disappointing agent pilot: the demo ran three steps, the real task needs thirty.

The Math Is Optimistic, and There Is a Paper Proving It

p^n assumes each step succeeds independently. That assumption is false, and the direction it is false in is the bad one.

The Illusion of Diminishing Returns (Sinha et al., 2025) ran a controlled experiment varying the error rate visible in an agent's own history and found self-conditioning: when a model's context contains its own prior mistakes, its per-step error rate measurably rises. Failure is not memoryless. An agent that has stumbled is more likely to stumble again, because the stumbles are now in its context.

Two consequences worth internalizing:

• Real compounding is worse than p^n predicts, because p is not constant — it degrades along the trajectory.
• Self-conditioning does not go away with model scale alone. The same paper found that without chain-of-thought, even frontier non-reasoning models failed after roughly four sequential steps, while the reasoning variant executed over 100 steps correctly. Reasoning-before-acting is what breaks the spiral, not a bigger model.

That last point is the practical one: if your agent degrades over long runs, adding reasoning budget will do more than upgrading the model.

The engineering responses, in order of effectiveness:

1. Use fewer steps. Collapse multi-step tool sequences into single higher-level tools. A tool that does the whole job in one call has one chance to fail, not six.
2. Verify after each step. A check that catches an error immediately stops it compounding — and stops it entering the context where it triggers self-conditioning.
3. Make recovery cheap. Retrying one failed step is affordable; restarting a thirty-step chain is not.
4. Give it room to reason. The 4-steps-vs-100-steps result above is the largest single lever in this list.
5. Then improve per-step reliability. It matters, but the exponent punishes you regardless.

The Rest of the Agent Failure Surface

Tool Calls Fail More Than You Think, and It Depends on the Model

The largest measurement available analyzed 78,295 real tool calls and found an aggregate failure rate of 70.8% — but the aggregate hides the real finding, which is how sharply it varies by model (arXiv:2602.04234):

A 6× difference in failure rate, and a qualitative shift in what fails. Weak models cannot form the call; strong models form it correctly and ask the wrong thing. Those need completely different fixes — schema constraints and retries for the first, better tool descriptions and verification for the second.

On the GAIA benchmark, failure decomposition put limited tool capability at 32.7% and planner error at 21.2% as the two largest buckets. A separate analysis of failed GAIA traces attributed 45.0% to tool-usage errors versus 36.7% to wrong final answers despite a correct process.

Read together: your agent's tools are more likely to be the problem than your agent's reasoning.

Silent wrong-completion is the one to design against hardest. Every other failure produces a signal — an error, a timeout, a bill. This one produces a confident report of success. If you take one operational practice from this article, make it: never let the agent be the sole judge of whether it succeeded.

The Agent's State Machine

Note the Verifying state. Many agent implementations go straight from Observing to Done. That missing state is exactly where silent wrong-completion lives.

When to Stop at Rung 3

Stop here when one agent with a good toolset closes the loop on your task. Do not climb to rung 4 to fix an unreliable agent — coordination multiplies unreliability rather than cancelling it. Get one agent completing tasks consistently first. Our field guide on agent evals covers how to know when you are actually there.

Rung 4: Agentic AI

Putting one or more agents to work on a shared objective is what makes a system agentic. It is achieved through planning, tool use, and feedback — with an orchestration layer added to coordinate work across multiple agents and workflows. Agents read and write shared task state and access tools, data, and the environment.

Two things are genuinely new here, and neither is "more agents":

1. An orchestrator that decides what work goes to whom and replans when progress stalls.
2. Shared task state that several agents read and write, which is what lets them build on each other rather than duplicating work.

A single agent inside an orchestrated system with shared state is agentic. Ten agents with no orchestrator and no shared state are just ten agents.

Multi-Agent Topologies

The Coordination Tax

Anthropic, writing about its own multi-agent research system, has been explicit that multi-agent architectures burn dramatically more tokens than single-threaded chat, and that the approach only pays off for tasks valuable enough to justify it. Their framing is worth internalizing: multi-agent systems work best when the task parallelizes genuinely and the value of the output is high enough to absorb the token cost.

Error propagation deserves a specific warning. In a single agent, a bad intermediate result is at least visible in the same trajectory. Across agents, agent A's hallucination arrives at agent B formatted as a clean, authoritative input — and B has no reason to doubt it. Cross-agent outputs should be treated as untrusted input, exactly as you would treat data crossing any other process boundary.

When Rung 4 Is Actually Right

• The work genuinely parallelizes — several independent lines of inquiry that combine at the end
• You need specialists with different tools and different instructions, not one generalist
• There is a real orchestrator and evaluator, not just a group chat
• One agent is already reliable on its portion
• The output is valuable enough to absorb the coordination cost

If you cannot tick all five, the honest answer is a workflow that calls one agent.

The Strongest Case For Climbing

This article argues for restraint, so it owes you the best counter-evidence.

On TravelPlanner, a multi-constraint itinerary benchmark, Planning with Multi-Constraints via Collaborative Language Agents (Zhang et al., Huawei Noah's Ark Lab, COLING 2025) measured a coordinated multi-agent framework at 42.68% success versus a single-agent GPT-4 + ReAct baseline at 2.92% — a more than 14× improvement. Read the condition, though: those are the hint-assisted numbers. Without hints the same table reports 22.40% versus 0.65% — a far larger ratio on a far smaller absolute base. Some tasks genuinely do not yield to one agent, no matter how well prompted.

And Anthropic's 90.2% improvement over single-agent Opus 4 on research tasks is real, measured, and came from exactly the architecture this section warns about.

The honest synthesis: rung 4 is not wrong, it is expensive and frequently misapplied. When the task genuinely parallelizes and the output is valuable, the multiplier pays for itself several times over. The failure is not choosing rung 4 — it is choosing it by default, for a task that never needed it, before rung 3 was working.

Agentic RAG: Where Rungs 2 and 3 Merge

The clean ladder blurs at one point worth naming, because it is where a large share of 2026 production systems actually live.

Agentic RAG puts the model in charge of retrieval rather than treating retrieval as a fixed pre-step.

The capability unlock is real: "which customers on the legacy plan also filed a ticket last quarter" requires two dependent lookups and plain RAG cannot do it. The tradeoff is equally real: your flat-cost component just became a variable-cost component.

The Named Mechanisms

"Agentic RAG" is not one technique. Three distinct mechanisms carry most of the value, each with a primary source:

Self-RAG's design targets a specific classic-RAG failure directly: indiscriminately retrieving a fixed number of passages whether or not retrieval is needed. If your RAG system retrieves on every query including "hello," that is the failure it fixes.

Treat agentic RAG as rung 3 applied to retrieval, not as an upgrade to rung 2, and budget accordingly.

What It Actually Costs: Real Numbers

Almost everything written about these four tiers describes cost qualitatively — "higher processing load," "more expensive." Here is what has actually been measured, with sources.

RAG vs Agentic Retrieval, Same 100 Queries

The cleanest like-for-like comparison available ran 100 queries through both a RAG pipeline and an MCP-based agentic retrieval system on the same corpus (Infragistics, July 2026):

The agentic system is better — 52% vs 38% win rate, twice the citations. It also costs 36× more per query and takes nearly 5× longer to produce its first token. That is the tradeoff in one table, and it is the table the rest of this cluster does not publish.

Note the TTFT column especially. A 29-second wait before the first token appears is a different product, not just a slower one.

Token Multipliers, From the People Who Ship It

Anthropic has published figures from its own production systems at two different moments, against two different baselines. They are often merged online. Do not merge them:

Two more findings from the same June 2025 writeup are worth more than the multipliers:

• Their multi-agent system outperformed single-agent Claude Opus 4 by 90.2% on an internal research eval. The extra cost bought something real.
• Token usage alone explained 80% of the performance variance on BrowseComp. Which means much of what looks like architectural cleverness is, statistically, just spending more.

And a caveat they state plainly: "upgrading to Claude Sonnet 4 is a larger performance gain than doubling the token budget on Claude Sonnet 3.7." A better model is often a cheaper lever than a bigger architecture.

Where RAG Latency Actually Goes

A published p95 latency budget for a production RAG system targeting a 3-second p95 (technovice.net):

  RAG p95 LATENCY BUDGET — target 3,000ms
  ────────────────────────────────────────────────
  query embedding             50ms   ▏
  vector search + filters    700ms   ▏▏▏▏▏▏▏
  reranking                  250ms   ▏▏▏
  prefill to first token     900ms   ▏▏▏▏▏▏▏▏▏
  decode + render          1,000ms   ▏▏▏▏▏▏▏▏▏▏
  headroom                   100ms   ▏
  ────────────────────────────────────────────────
  adding RAG roughly DOUBLES time-to-first-token
  (495ms → 965ms in the measured baseline)
  ⚠ vector-index p95 measured at 64× the median
— the tail is the whole problem

That last line is the one to act on. Your median RAG query is fine. Your p95 is where the user leaves.

Two Baselines, Not One Range

The same discipline applies here that applies to the Anthropic table above: these figures are measured against different baselines, and merging them is the most common error in this literature.

Infragistics ran the one genuine agent-vs-RAG comparison available: 36× cost, 2.5× latency, for a 14-point win-rate gain. Separate token-consumption analyses put agent tasks at 5–30× a standard chat exchange — a different baseline, not a RAG query. Nothing in either study establishes what RAG costs relative to chat in the same rig, so the two cannot be folded into a single "agent vs RAG" multiplier.

Budget from the Infragistics number if you are moving from RAG to an agent. Treat the chat-baseline range as directional corroboration, not as a second measurement of the same thing. And note that none of these are median figures you can plan capacity from — they are averages over distributions with long tails.

Why Multi-Agent Systems Fail

When rung 4 breaks, it usually is not the model's fault — and there is now a proper taxonomy saying so.

Berkeley researchers hand-annotated 150 multi-agent failure traces (κ = 0.88 inter-annotator agreement), then scaled the analysis to 1,600+ traces across seven frameworks, producing MAST, a 14-mode taxonomy (Why Do Multi-Agent LLM Systems Fail?):

The paper's central claim is the one that should change how you plan: "improvements in the base model capabilities will be insufficient to address the full taxonomy."

Waiting for a better model will not fix your multi-agent system. Nearly 42% of failures are specification and design problems — things you wrote, or failed to write. Another 37% are agents talking past each other. Only the remaining fraction looks anything like a capability limit.

The Rungs Nest Inside Each Other

One thing the ladder metaphor hides: these are not mutually exclusive. Real systems compose them, and knowing where the nesting stops is its own decision.

An agent whose third tool is a RAG pipeline is completely normal and usually correct. A RAG pipeline is a well-behaved tool: bounded latency, bounded cost, one call, one result.

The stopping rule: nest downward freely — an agent may call a workflow, a workflow may call RAG, RAG calls an LLM. Be extremely reluctant to nest upward or sideways — an agent that calls another agent that calls another agent compounds every failure mode in this article at every level, and the resulting trace is close to un-debuggable.

One layer of agent-calling-agent, with an orchestrator that owns the plan, is the practical ceiling for most teams.

The Decision Procedure

Ask these in order. Stop at the first yes.

As a checklist you can keep next to you:

  1. Can a single well-written prompt do it?                → LLM.       Stop.
       ↓ no
  2. Does it only need facts you own, fetched once?         → RAG.       Stop.
       ↓ no
  3. Can you write the steps down in advance as code?       → WORKFLOW.  Stop.
     (chaining, routing, parallel, evaluator-optimizer)        ← most teams belong here
       ↓ no — the path genuinely varies per input
  4. Does ONE agent with tools close the loop?              → AI AGENT.  Stop.
       ↓ no
  5. Genuine parallel specialists + shared state, and you
     can afford coordination cost AND concurrent debugging? → AGENTIC AI.

The Rule Behind the Procedure

Do not add a rung to fix a bug in the rung below.

An agent will not fix bad retrieval — it will retrieve badly several times and cost more. A second agent will not fix a flaky first agent — it will inherit the flakiness and add coordination on top. Fix the layer that is broken.

What Each Rung Is Best At

Each rung wins a narrow class of job and wastes money on every other one. The cost gap between adjacent rungs is large enough to matter — the one measured agent-vs-RAG comparison in this article puts it at 36× — so matching the rung to the shape of the task is worth more than any prompt optimization you will do afterward. Match on task shape first, team enthusiasm second.

Two Axes Nobody Tabulates: Memory and Governance

Two dimensions change as sharply as cost does, and almost no comparison of these tiers writes them down.

Where State Lives

The last cell is the one that bites. "Several agents, concurrently" is a concurrency problem, and most agent frameworks give you a key-value store with last-write-wins semantics and no conflict detection. If two agents update the same plan field, one of them silently loses. Design for that before you hit it.

Governance and Control Points

Each rung moves the security boundary, and the questions your compliance reviewer asks change with it.

The rung-2 row is worth pausing on. RAG creates a permissions problem that a plain LLM does not have: if your index contains documents some users cannot see, retrieval will happily surface them. Document-level ACLs must be enforced at retrieval time, not at ingestion time, and this is one of the most common production security gaps in RAG systems.

At rungs 3 and 4 the question shifts from what did it read to what did it do — a materially harder thing to audit, and the reason approval gates on destructive tools are not optional.

A Worked Example: One Support Pipeline, Four Rungs

Abstract ladders are easy to agree with and hard to apply. Here is one concrete system, and where each rung is the right answer.

The system: a customer support pipeline.

Most tickets never leave the first two rows. That distribution is the entire argument of this article in one table: the expensive rungs should handle the tail, not the head.

Quality Per Dollar

The metric that makes this concrete is quality per dollar, not quality. Using the measured figures from earlier: if agentic retrieval wins 52% of answers versus RAG's 38%, that is a 1.37× quality gain for a 36× cost. On a high-volume FAQ, that trade is indefensible. On a contract review worth thousands, it is obviously correct.

The same architecture is right and wrong depending on the value of the task. Decide per stage, not per system.

Evaluation Gets Harder at Every Rung

This is the hidden tax, and the strongest practical argument for climbing reluctantly.

What to Actually Measure at Rung 2

Ragas (Es et al., EACL 2024) formalized RAG evaluation into three reference-free metrics, and the split is the useful part:

The paper's own finding is a warning: context relevance is the hardest of the three to evaluate, partly because the judge model itself struggles to pick the crucial sentences out of a long context.

Sit with that. At rung 2 — the second rung — the evaluator is already hitting the same long-context limitation as the system it is evaluating. Evaluation difficulty is compounding before you have even reached agents.

Why Rung 4 Evaluation Is Qualitatively Different

The MAST work is itself evidence. To get a reliable failure-attribution signal across multi-agent traces, the researchers needed six expert human annotators plus an LLM judge, validated against human agreement at κ = 0.88. Standard "did the task succeed" scoring could not localize which agent, at which turn caused the failure.

That is the real cost of rung 4: you stop scoring a model's output and start scoring a system's process, and the tooling for that barely exists.

⚠️ Before you trust any improvement you measure at rung 3 or 4, establish your run-to-run variance first. In our own measurements, running the same prompt through the same rig twice moved per-shape results by 10.5% to 50.0%, mean 29.6%, while aggregate totals stayed far tighter at 5.2%. Most single-run agent improvements sit inside a band like that. We wrote up the method and three other negative results in A Year of Agent Memory Experiments.

How to Climb Safely

If you have established that you genuinely need the next rung, climb with these in place.

Step 3 is non-negotiable. An agent that grades its own homework will pass.

Climbing Down Is Also an Option

The move nobody writes about: replacing an agent with a workflow once you learn what it actually does. Run an agent in development to discover the real decision paths, observe that 90% of runs take one of three routes, then write those three routes as code and keep the agent only for the long tail. You get the reliability of rung 2.5 for the common case and the flexibility of rung 3 for the rest.

Common Mistakes

Most production failures at these tiers trace to an architecture chosen before the problem was understood, not to model quality. The MAST taxonomy makes the point quantitatively: 41.77% of multi-agent failures are specification and design problems, and another 36.94% are inter-agent misalignment — both decided at architecture time, not at inference time. The eight mistakes below are the recurring shapes.

How This Maps to Taskade

Every rung on this ladder exists as a surface you can build on, which is unusual — most platforms make you pick one rung and commit.

Working examples of each rung, built and published by other people, are browsable in the App Kits gallery — useful for seeing which rung a given problem actually landed on before you pick one yourself.

The architecturally interesting part is what the shared task state is. In most orchestration frameworks it is a store only the system can read. In Taskade it is an ordinary project — same views, same editing, same sharing a person uses.

That means the "shared state" box in the rung-4 diagram is something a human can open and correct mid-run. When an agent writes a wrong assumption into the shared plan, you fix the line rather than restarting the run. It also means the record of what the system did is legible afterward, which is what makes rung-4 debugging tractable at all.

To be precise about the limits: this is coordination over shared state with an evaluator, not arbitrary-topology orchestration, and there is no automated self-improvement loop. Those are different claims and we are only making the first one.

Which Framework Belongs at Which Rung

Tool choice follows the rung, not the other way around. Picking an orchestration framework before you know your rung is how teams end up with a multi-agent graph running a task that needed one prompt.

Two notes worth having:

• MCP sits across rungs 3 and 4, not above them. It standardizes how an agent reaches tools and data. It is plumbing for the rung you chose, not a rung of its own — though it increasingly appears in these comparisons as if it were.
• A graph framework does not make you agentic. LangGraph models systems as an explicit graph of nodes with shared state — which is an excellent way to build a workflow. Using a graph framework to run a fixed sequence is rung 2.5 with better tooling, and that is a perfectly good place to be.

The Academic Anchor

If you want one peer-reviewed reference for the agent-versus-agentic distinction, it is AI Agents vs. Agentic AI: A Conceptual Taxonomy, Applications and Challenges (Sapkota, Roumeliotis & Karkee, Information Fusion, 2025). It characterizes agents as modular LLM-driven systems for task-specific automation, and agentic AI as a shift marked by "multi-agent collaboration, dynamic task decomposition, persistent memory, and coordinated autonomy" — then names the failure classes each paradigm carries, including coordination failure and emergent behavior.

It is the closest thing this topic has to a canonical citation, and remarkably few of the pages ranking for these terms link to it.

The Short Version

1. The four architectures differ by who decides how many model calls happen.
2. Each rung adds a capability and a failure mode.
3. Errors compound at least as fast as p^n. A 95%-reliable step is a 36% task at twenty steps — and self-conditioning makes the real curve worse than the formula, not better.
4. Most RAG failures are retrieval failures. Run the taxonomy before blaming the model.
5. The workflow rung is the one everyone skips and where most systems belong.
6. Never let an agent be the sole judge of its own success.
7. Do not add a rung to fix a bug in the rung below.
8. Evaluation difficulty grows faster than capability. Measure your noise floor first.

▲ ■ ●  Memory, Intelligence, Execution — pick the rung your problem needs, not the one the industry is selling.

Build on the right rung →

Deeper Reading

Primary sources

• Lewis et al., Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks, arXiv:2005.11401, 2020
• Yao et al., ReAct: Synergizing Reasoning and Acting in Language Models, arXiv:2210.03629, 2022
• Shinn et al., Reflexion: Language Agents with Verbal Reinforcement Learning, arXiv:2303.11366, 2023
• Sumers et al., Cognitive Architectures for Language Agents (CoALA), arXiv:2309.02427, 2023
• Anthropic, Building effective agents — the workflows-vs-agents distinction
• Anthropic, Effective context engineering for AI agents
• Chroma, Context Rot — 18 models, 194,480 calls

On this blog

• What Are AI Agents? — the full primer on rung 3
• Agentic Workflows Explained · The 21 Agentic Design Patterns
• Single Agent vs Multi-Agent AI Teams · Best Practices for Multi-Agent AI Teams
• What Is an AI Agent Harness? · What Are AI Agent Evals?
• Context Engineering: The Complete Field Guide · Fine-Tuning vs RAG
• A Year of Agent Memory Experiments · Structure Beats Instruction
• Multi-Agent Systems in Production · AI Agent Reliability · AI Agent Error Recovery
• Vector Databases and Embeddings Explained · How Do LLMs Work?

Concepts

• Large Language Models · Retrieval-Augmented Generation · Agentic RAG
• Agentic AI · Multi-Agent Systems · Agent Orchestration
• Agent Loop · Agent Harness · ReAct Pattern
• Context Window · Context Rot · Vector Database · Embeddings
• Evals · Agent Evaluation · Non-Determinism

Frequently Asked Questions