Definition: Channel capacity is the maximum rate at which information can be sent over a noisy communication channel with an error rate as close to zero as you want. Shannon proved in 1948 that below this rate, reliable communication is always achievable, and above it, it is impossible.
TL;DR: Noise sets a speed limit, not an accuracy limit. Below capacity you can drive errors arbitrarily close to zero by adding redundancy. That single result is why digital replaced analog in storage, transmission, and computing. Build an AI app free →
What Is Channel Capacity?
Every real channel corrupts what passes through it. A phone line adds hiss, a disk flips bits, a radio link fades. Before 1948, the working assumption among engineers was that noise forced a permanent trade: send slower and more carefully to get fewer errors, or send faster and accept more.
Shannon proved that assumption wrong. Every channel has a number, its capacity C, measured in bits per second or bits per use. The behavior on either side of that number could not be more different.
It is a cliff, not a slope. That discontinuity is what made the result famous.
The Noisy-Channel Coding Theorem
Shannon's theorem has two halves, and both are surprising.
- Achievability. For any rate below capacity and any target error rate, however small, there exists a code that achieves it.
- Converse. For any rate above capacity, no code achieves an arbitrarily small error rate.
The achievability half was the shock. Engineers expected that pushing errors toward zero would require pushing the rate toward zero. Shannon showed you keep a fixed, positive rate and still get vanishing errors, by encoding long blocks of symbols together rather than protecting each symbol individually.
The catch is that his proof is non-constructive. It proves good codes exist without saying how to build one. Closing that gap took roughly fifty years of coding theory, from Hamming codes through Reed-Solomon to turbo codes and LDPC codes, which finally approach the Shannon limit in practice.
Redundancy Is the Mechanism
Capacity is bought with redundancy: sending more symbols than the message strictly needs, arranged so that corrupted symbols can be detected and repaired.
NAIVE REPETITION BLOCK CODING
---------------- ------------
Send "1" as "111" Encode 4 data bits + 3 check bits
One flip -> majority vote Any single flip -> located and fixed
Rate: 1/3 Rate: 4/7
Fixes 1 error per 3 bits Fixes 1 error per 7 bits
Both work. The second is far more efficient,
because the check bits protect the block, not each bit.
Naive repetition works but wastes most of the channel. Real codes spread the protection across a block so that a small number of extra symbols guards a large number of real ones. Shannon's theorem says how far that trick can be pushed.
Reliability From Unreliable Parts
The deepest consequence is architectural. If you can make errors arbitrarily rare at a fixed cost, you can build a reliable system out of unreliable components, which is the founding assumption of all digital technology.
| Domain | Unreliable part | What capacity buys |
|---|---|---|
| Storage | Disks and flash cells that fail | Error-correcting codes, RAID, checksums |
| Networking | Lossy, noisy links | Retransmission, forward error correction |
| Memory | Cosmic-ray bit flips | ECC RAM in every server |
| Deep space | Signals near the noise floor | Codes that recover images from Voyager |
| Wireless | Fading and interference | The rate ladder in every Wi-Fi and 5G standard |
This is also why analog lost. An analog signal accumulates noise at every hop and can never be cleaned. A digital signal below capacity can be regenerated perfectly at every hop, so a thousand-hop path is as clean as a one-hop path. Every copy of a digital file is a perfect copy. That property is entirely a consequence of this theorem.
Bandwidth, Noise, and the Shannon-Hartley Formula
For the common case of a channel with bandwidth B and a signal-to-noise ratio S/N, capacity is:
C = B × log2(1 + S/N)
Two readings follow immediately, and both match everyday experience.
| Lever | Effect on capacity | Practical note |
|---|---|---|
| Double the bandwidth | Roughly doubles capacity | Linear, and why spectrum is valuable |
| Double the signal power | Adds less than one bit per symbol | Logarithmic, so power has sharply diminishing returns |
This is why modern wireless standards chase wider channels and more antennas rather than simply transmitting louder. Power is the expensive lever with the worst return.
The Same Shape in AI Systems
Shannon's framing transfers cleanly to systems that are not radios, and the analogy is useful when reasoning about agent evaluation.
- A model is a noisy channel. It maps intent to output with a nonzero error rate. You cannot make an individual call perfect.
- Redundancy is the fix, not more power. Running a check step, a second opinion, or a validator recovers reliability the same way error-correcting codes do. This is the mechanism behind self-consistency, LLM-as-a-judge, and guardrails.
- There is a capacity you cannot exceed. If a task genuinely needs information the model does not have, no amount of retrying supplies it. That is a capacity problem, and the fix is retrieval or context, not repetition.
The engineering discipline is the same one Shannon established: stop trying to make the component perfect, and design the system so that imperfect components compose into a reliable result.
Building Reliability Into Your Own Process
The workflow version of redundancy is a check step: a second pass that catches the small fraction of items the first pass got wrong, without a human reviewing all of them.
Describe it to Taskade Genesis: "an intake board where every submission gets scored by an agent, and anything below a confidence threshold is routed to a human queue." Taskade EVE assembles it as living software, and a reliable automation moves items between the lanes. You get near-zero escaped errors without checking every row yourself.
Related Concepts
- Information Theory: the field this theorem anchors
- Entropy: the compression floor, capacity's counterpart
- Compression and Intelligence: the other half of Shannon's legacy
- Agent Evaluation: the same problem in agent systems
- Guardrails: redundancy applied to model outputs
- Self-Consistency: sampling several times and taking the agreement
- Von Neumann Architecture: the other 1940s design still in everything
Frequently Asked Questions About Channel Capacity
What is channel capacity in information theory?
Channel capacity is the maximum rate, in bits per second or bits per channel use, at which information can be transmitted over a noisy channel with an error rate approaching zero. Shannon defined and proved it in 1948.
What is Shannon's noisy-channel coding theorem?
It states that below a channel's capacity, codes exist that make the error rate arbitrarily small, and above capacity no such code exists. It replaced the assumed speed-versus-accuracy trade with a hard threshold.
Why is the noisy-channel theorem important?
It proved you can build reliable systems from unreliable parts. That underpins error-correcting codes, digital storage, networking, and the replacement of analog by digital everywhere, since a digital signal below capacity can be regenerated perfectly at every hop.
What is the Shannon-Hartley formula?
C = B × log2(1 + S/N), where B is bandwidth and S/N is the signal-to-noise ratio. Capacity scales linearly with bandwidth but only logarithmically with power, which is why wireless standards pursue wider channels rather than louder transmission.
Does Shannon's theorem tell you how to build a good code?
No. The proof is non-constructive: it shows good codes exist without describing them. Building codes that approach the limit took decades, culminating in turbo codes and LDPC codes.
Can you exceed channel capacity?
Not with reliable communication. You can transmit above capacity, but the error rate cannot be driven to zero no matter what code you use. In practice systems back off below capacity and spend the difference on redundancy.
How does channel capacity apply to AI agents?
A model behaves like a noisy channel: individual calls have a nonzero error rate. Reliability comes from redundancy such as validation steps, second opinions, and guardrails, rather than from expecting any single call to be perfect. But if the needed information is absent entirely, that is a capacity limit no retry fixes.
What is the difference between entropy and channel capacity?
Entropy bounds compression: the fewest bits needed to represent a source. Capacity bounds transmission: the most bits a channel can carry reliably. One is the floor on the source side, the other the ceiling on the channel side.
Further Reading
- Claude Shannon and the Invention of the Bit: the 1948 paper and its aftermath
- Compression Is Intelligence: Shannon's other theorem in modern form
- Foundations: the rest of the theory under modern AI
- Agent Evaluation: reliability engineering for AI systems