Definition: A world model is an internal representation of how an environment responds to actions. An agent with one can simulate the consequences of a plan before committing to it, which means it can learn from imagined experience instead of paying for every lesson in the real world.
TL;DR: A world model lets an agent rehearse. Instead of trying something expensive to find out what happens, it predicts the outcome internally and acts only on the plan that survives. The idea runs from Sutton's Dyna (1990) through World Models (2018) to today's agent research. See how recorded history became one.
World Model Explained in 3 Levels
Level 1, the everyday version. Before you move a sofa through a doorway, you picture whether it will fit. You do not try every angle physically. That mental picture is a world model.
Level 2, the builder's version. An agent that knows "calling this tool with a bad path returns an error" can avoid the call. A model of consequences lets it skip experiments whose outcomes it can already predict.
Level 3, the architect's version. A learned dynamics model maps state and action to a predicted next state and reward, usually in a compressed latent space. A policy can then be optimized against imagined rollouts, decoupling sample efficiency from real-environment interaction cost — at the price of compounding model error over long horizons.
Why World Models Matter
Real experience is expensive. Every real attempt costs time, money, or an irreversible consequence. A world model converts some of that cost into computation.
The Lineage
| Year | Work | Contribution |
|---|---|---|
| 1990 | Sutton, Dyna | Learning, planning and acting in one architecture, with planning run against a learned model |
| 2018 | Ha & Schmidhuber, World Models | Train a compact environment model, then train the policy entirely inside its generated "dream" |
| 2019 | Hafner et al., Dream to Control | Improve behavior by backpropagating through imagined latent rollouts |
| 2023 | DreamerV3 | One configuration mastering many domains through a learned world model |
| 2026 | Dream-RSI | Uses recorded history directly as an exact model instead of learning an approximate one |
Learned Model vs Recorded History
Not every "model of the environment" has to be learned. A replay simulator uses an exact record of what already happened.
| Learned world model | Recorded history | |
|---|---|---|
| Fidelity | Approximate | Exact |
| Coverage | Can generalize beyond observed data, less reliably out of distribution | Only what was visited |
| Error over long horizons | Compounds | None |
| Cost to obtain | Training | Free, as a by-product |
The trade is coverage against exactness. A learned model can imagine branches nobody explored, and may be wrong about them. A recorded history is never wrong and cannot leave the map.
Limitations
- Compounding error. Small prediction errors accumulate across an imagined rollout, so long-horizon plans drift from reality.
- Distribution shift. A model is only trustworthy where it has seen data. Novel situations are exactly where it is least reliable and most needed.
- Objective mismatch. A model trained to predict observations may be accurate about irrelevant detail and wrong about the few variables that decide the outcome.
- Verification is still required. A plan that succeeds in imagination has not succeeded. The loop is only sound when real execution confirms it.
Related Concepts
- Replay Simulator · Recursive Self-Improvement · Off-Policy Evaluation
- Reinforcement Learning · Planning and Reasoning · Test-Time Compute
- Agent Loop · Agent Memory · Evals
In Taskade, the practical version of this is simpler than a learned model: projects hold what the system knows, and automations act on it, so the "model of the environment" is just the workspace itself.
Read next: Replay Simulators Explained — what happens when the "model" is a literal record of past work.