Definition: A replay simulator is a completed run treated as an environment. If a system records each decision alongside the outcome that decision actually produced, an alternative strategy can be scored by walking that record in a different order and reading the stored results — instead of paying to execute the whole process again.
TL;DR: A replay simulator is history you can act inside. Because every outcome is already stored, testing a new strategy costs a read rather than a run. The term was popularized by Dream-RSI (Google and Google DeepMind, September 2026), which reports up to 162× fewer agent calls on one benchmark. See it explained in full.
Replay Simulator Explained in 3 Levels
Level 1, the everyday version. You already walked the maze once and drew a map. Planning a better route does not require walking every corridor again — you reason over the map.
Level 2, the builder's version. Your agent tried twelve things last week and you recorded what each one produced. You can now ask "what if it had stopped after four?" and answer it by reading the log, at no model cost.
Level 3, the architect's version. A discovery tree where each node stores its parent, its artifact and its score is an empirical model of the visited subset of the search space. An alternative exploration policy induces a different trajectory through that tree, and its value is computed from stored outcomes — an off-policy evaluation with zero execution cost and zero model error.
How a Replay Simulator Works
The loop alternates between acting in the world and evaluating inside the record.
Nobody builds the simulator. It accumulates as a by-product of doing the work.
What Makes a Record Replayable?
Three properties, all of them required.
| Property | What it means | Fails without it |
|---|---|---|
| Structured | Parent and child relationships, not a flat log | You cannot re-order a list of events into a different path |
| Carries outcomes | Each decision stored next to what it produced | You can replay the choices but learn nothing about results |
| Re-walkable | Can be visited in an order nobody used | You can only re-read the original sequence |
A chat transcript typically fails the first. An embedding in a vector database fails all three, because "what happened next" was never a field. A tree of attempts with scores attached satisfies all three.
This is why legibility is not an aesthetic preference in agent memory design. A readable, structured record of decisions and their results is replayable. A compressed, opaque one is not.
Replay Simulator vs World Model
They belong to the same family and make opposite trades.
| Learned world model | Replay simulator | |
|---|---|---|
| How it is obtained | Trained on collected experience | Accumulates for free |
| Fidelity | Approximate, error compounds | Exact, no model error |
| Coverage | Can generalize beyond collected data, less reliably out of distribution | Only what was visited |
| Can imagine unseen branches | Yes | No |
You trade coverage for exactness, and the model costs nothing.
Where the Idea Comes From
- 1990 — Dyna (Sutton): learning, planning and acting in one loop, with planning run against a model rather than the world.
- 2018 — World Models (Ha & Schmidhuber): train a compact environment model, then train the policy inside its "dream."
- 2019–2025 — Dreamer (Hafner et al.): improve a policy by imagining latent rollouts.
- 2026 — Dream-RSI: apply the pattern at the meta level, where the "environment" is literal recorded history and the thing improved is the exploration strategy.
The Main Limitation
A replay can only answer questions about territory that was recorded. If a better strategy would have opened a branch nobody opened, no outcome is stored and the replay cannot say anything about it.
This is why the loop must alternate: replay offline to choose a strategy, then go online to discover new ground. A replay simulator is excellent for deciding how to order, batch and stop across known territory, and useless for finding unknown territory.
What You Can Replay Without a Research Budget
The entry cost is a logging decision, not a modeling one. If your agent records what it tried, in what order, and what each attempt produced, you can already replay:
- A different stopping rule — would a lower step cap have cut any completed work?
- A different retry cap — did attempts 3 and 4 ever produce anything useful?
- A different batching strategy — which independent calls could have run together?
- A different escalation threshold — when should this have gone to a larger model?
The hard part is rarely the replay. It is that most systems store the decision in one place and the outcome in another, with no key joining them.
Related Concepts
- World Model · Recursive Self-Improvement · Off-Policy Evaluation
- Agent Memory · Agent Loop · Agent Harness
- Evaluation Noise Floor · Evals · Non-Determinism
- Reinforcement Learning · Exploration and Discovery
Taskade keeps the record of how work was done in the same projects the work lives in, so an agent is grounded on something a person can open, correct and re-walk. Browse what people have built that way in the app gallery.
Read next: Replay Simulators Explained — the full walkthrough, the reported results, and the honest limits.