Skip to main content
Introducing TSK-1Introducing TSK-1·Taskade's intelligence layer.
taskade
Create
Connect
Clone
PricingHelpDashboard →Dashboard →
PricingLoginSign up for free →Sign up for free →
Dashboard →Dashboard →
Sign up for free →Sign up for free →
Loved by 1M+ users·Hosting 150,000+ apps·Deploying 500K+ AI agents·Running 1M+ automations·Backed by Y Combinator·Powered by TSK-1
Taskade
AppsTSK-1AI ModelsBuilderPricingFeatures
See all▾
Contact usIntegrationsMCP ServerAgent skillllms.txtPressAboutConnectProductivityVideosReviewsFAQ
Learn
Taskade GenesisProjectsAI AgentsAutomationConnectorsVideo Tutorials
See all▾
Import & ExportSearch ArticlesAccount & Billing
App Factory
FeaturedDashboardsOperations
See all▾
CommerceFinanceSalesCRMsInventoryPortalsInvoicesBookingToolsCreators
Downloads
Mobile appsDesktop appsAndroid
See all▾
iOSMacWindowsChromeFirefoxEdge
Compare
vs Cursorvs Boltvs Lovable
See all▾
vs V0vs Windsurfvs Replitvs Emergentvs Devinvs Claude Codevs ChatGPTvs Claudevs Perplexityvs GitHub Copilotvs Figma AIvs Notionvs ClickUpvs Asanavs Mondayvs Trellovs Jiravs Linearvs Todoistvs Evernotevs Obsidianvs Airtablevs Basecampvs Mirovs Slackvs Bubblevs Retoolvs Webflowvs Framervs Softrvs Glidevs FlutterFlowvs Base44vs Adalovs Durablevs Gammavs Squarespacevs WordPressvs UI Bakeryvs Zapiervs Makevs n8nvs Jaspervs Copy.aivs Writervs Rytrvs Manusvs Crewvs Lindyvs Relevance AIvs Wrikevs Smartsheetvs Monday Magicvs Codavs TickTickvs Any.dovs Thingsvs OmniFocusvs MeisterTaskvs Teamworkvs Workfrontvs Bitrix24vs Process Streetvs Toggl Planvs Motionvs Momentumvs Habiticavs Zenkitvs Google Docsvs Google Keepvs Google Tasksvs Microsoft Teamsvs Dropbox Papervs Quipvs Roam Researchvs Logseqvs Memvs WorkFlowyvs Dynalistvs XMindvs Whimsicalvs Zoomvs Remember The Milkvs Wunderlist
AI Agents
FeaturedProject ManagementProductivity
See all▾
MarketingTranslatorContentWorkflowResearchPersonalOperations IntelligenceSalesSocial MediaTo-Do ListCRMTask AutomationCoachingCreativityTask ManagementBrandingFinanceLearning and DevelopmentBusinessCommunity ManagementMeetingsAnalyticsDigital AdvertisingContent CurationKnowledge ManagementProduct DevelopmentPublic RelationsProgrammingHuman ResourcesE-CommerceEducationLegalEmailSEODeveloperVideo ProductionDesignFlowchartDataPromptNonprofitAssistantsTeamsCustomer ServiceTrainingTravel PlanningUML DiagramER DiagramMath TutorLanguage LearningCode ReviewerLogo DesignerUI WireframeFitness CoachLead EnrichmentFounder OSSales DevelopmentBookkeepingRecruitingWebsite MonitoringField ServiceLicensingAll Categories
Automations
FeaturedAI Workflow AutomationLogic Automations
See all▾
AI Agent AutomationTrigger AutomationsAgentic Process AutomationAction AutomationsAI Models in WorkflowsAgentic AutomationMulti-Agent AutomationSocial MediaBusiness-in-a-BoxOperations IntelligenceInvestor OperationsEducation & LearningHealthcare & ClinicsReal EstateStripeSalesHR & People OpsField Service & DispatchRenewals & LicensesE-commerceContentMarketingEmailCustomer SupportHubSpotProject ManagementAgentic WorkflowsAppointment SchedulingCalendarReportsSlackWebsiteFormTaskWeb ScrapingWeb SearchChatGPTText to ActionYouTubeLinkedInX (Twitter)GitHubDiscordMicrosoft TeamsWebflowIndustry News & RSS FeedsGoogle WorkspaceManufacturing & OperationsAI Agent TeamsNotion AutomationsProposalBookkeeping & ExpensesClient OnboardingGoogle SheetsGoogle DriveGoogle CalendarGoogle FormsShopifyAsanaAirtableTrelloTodoistMailchimpClickUpGoogle DocsGmailGoogle TasksJiraLinearMicrosoft OutlookTelegramCalendlyTypeformSalesforceMonday.comTwilioWhatsApp BusinessWordPressZoomHTTP and WebhooksMCP AutomationFacebook PagesRedditApolloPipedriveFigmaAttioBrevoCal.comConfluenceHighLeveliCal FeedJotformOutlook CalendarMicrosoft To DoPushoverResendSendGridSMTPSupabaseTallyAll Categories
Prompts
Client PortalsDocument IntakeField Service
See all▾
Operations IntelligenceCRMAdvertisingDesignInvoicing and PaymentsSocial MediaQuotes and EstimatesCalculatorsCodingBooking and SchedulingConsultingCustomer SupportComplianceTime TrackingRetail OperationsSaaS and SubscriptionsInsuranceGovernance and OversightNonprofitPatient IntakeBlog WritingBrandingPersonal FinanceHuman ResourcesPublic RelationsTeam CollaborationProduct ManagementSupportAgencyReal EstateMarketingResearchSalesCopywritingContentProject ManagementWebsite CreationStrategyE-commerceEngineeringSEOEducationEmail MarketingUX/UIProductivityInfluencer MarketingAnalyticsEntrepreneurshipLegalVibe CodingRecruitingAll Categories
Templates
FeaturedChatGPTTable
See all▾
PersonalProject ManagementSalesFlowchartTask ManagementEngineeringOperations IntelligenceInventory ManagementEducationDesignTo-Do ListMarketingMind MapGantt ChartOrganizationalPlanningMeetingsTeam ManagementStrategyGamingProductionProduct ManagementStartupRemote WorkY CombinatorRoadmapCustomer ServiceLegalEmailBudgetsContentConsultingE-CommerceStandard Operating Procedure (SOP)Human ResourcesProgrammingMaintenanceCoachingSocial MediaHow-TosResearchMusicTrip PlanningCRMClient OnboardingEmployee OnboardingSOPBug TrackerRecruitment TrackerFormSales PipelineContent CalendarMarketing PlanProduct RoadmapBusiness PlanSWOT Analysis30-60-90 Day PlanInterviewNotion AlternativeKPIStrategic PlanMeeting AgendaInvoiceRisk RegisterIT Asset ManagementKanban BoardChange ManagementCommunication PlanRFPScope of WorkStatement of WorkHelpdeskKnowledge BaseCreative BriefGoal SettingExecutive SummaryGap AnalysisBooking SystemEvent ManagementPortfolio TrackerCustomer Onboarding PortalsClient PortalAgency OperationsFinance TrackingAll Categories
Generators
AI SoftwareNo-Code AI AppAI App
See all▾
AI WebsiteAI DashboardAI FinanceAI Operations IntelligenceAI FormAI AgentAI Client Portal BuilderAI WorkspaceAI ProductivityAI To-Do ListAI WorkflowsAI EducationAI Mind MapsAI FlowchartAI Scrum Project ManagementAI Agile Project ManagementAI MarketingAI Project ManagementAI Social Media ManagementAI BloggingAI Agency WorkflowsAI ContentAI Software DevelopmentAI MeetingAI PersonasAI Outline GeneratorAI SalesAI ProgrammingAI DesignAI FreelancingAI ResumeAI Human ResourceAI SOPAI E-CommerceAI EmailAI Public RelationsAI InfluencersAI Content CreatorsAI Customer ServiceAI BusinessAI PromptsAI Tool BuilderAI SEOAI Gantt ChartAI CalendarsAI BoardAI TableAI ResearchAI LegalAI ProposalAI Video ProductionAI Health and WellnessAI WritingAI PublishingAI NonprofitAI DataAI Event PlanningAI Game DevelopmentAI Project Management AgentAI Productivity AgentAI Marketing AgentAI Personal AgentAI Business and Work AgentAI Education and Learning AgentAI Task Management AgentAI Customer Relations AgentAI Programming AgentAI SchemaAI Business PlanAI Pitch DeckAI InvoiceAI Lesson PlanAI Social Media CalendarAI API DocumentationAI Database SchemaAI Marketing PlanAI Sales Pipeline GeneratorAI Course BuilderInternal ToolsBooking SystemReal Estate CRMInventory ManagementAI CRM BuilderAI TimesheetAI DispatchAI NewsletterAI Clinic OperationsAI Directory BuilderAll Categories
Taskade AI
Video GuideAI App BuilderVibe Coding
See all▾
Agent BuilderDashboard BuilderCRM BuilderWebsite BuilderForm BuilderWorkflow AutomationWorkflow BuilderBusiness-in-a-BoxAI for MarketingAI for Developers
Converters
AI Featured ConvertersAI PDF ConvertersAI CSV Converters
See all▾
AI Markdown ConvertersAI Prompt to App ConvertersAI Data to Dashboard ConvertersAI Workflow to App ConvertersAI Idea to App ConvertersAI Flowcharts ConvertersAI Mind Map ConvertersAI Text ConvertersAI YouTube ConvertersAI Knowledge ConvertersAI Spreadsheet ConvertersAI Email ConvertersAI Web Page ConvertersAI Video ConvertersAI Coding ConvertersAI Task ConvertersAI Kanban Board ConvertersAI Notes ConvertersAI Education ConvertersAI Language TranslatorsAI Business → Backend App ConvertersAI File → App ConvertersAI SOP → Workflow App ConvertersAI Portal → App ConvertersAI Form → App ConvertersAI Schedule → Booking App ConvertersAI Metrics → Dashboard ConvertersAI Game → Playable App ConvertersAI Catalog → Directory App ConvertersAI Creative → Studio App ConvertersAI Agent → Agent App ConvertersAI Audio ConvertersAI DOCX ConvertersAI EPUB ConvertersAI Image ConvertersAI Resume & Career ConvertersAI Presentation ConvertersAI PDF to Spreadsheet ConvertersAI PDF to Database ConvertersAI PDF to Quiz ConvertersAI Image to Notes ConvertersAI Audio to Notes ConvertersAI Email to Tasks ConvertersAI CSV to Dashboard ConvertersAI YouTube to Flashcards ConvertersURL to NotesVideo → SummaryAI Receipts to Expense Tracker ConvertersAI Docs to Knowledge Base ConvertersAI Form to Client Portal ConvertersSpreadsheet to CRMAll Categories
Wiki
AI Agent ExamplesLiving DNAAutonomous Workspaces, Agents & Apps
See all▾
Taskade GenesisAI AgentsAutomationProjectsQuantum AI & Taskade Genesis QuantumFoundations: The Theory Under Modern AICognitive Science: Mind, Brain and AIPlatformAI InfrastructureIntegrationsProductivityMethodsProject ManagementAgileScrumAI ConceptsCommunityTerminologyFeatures
Docs
Getting StartedREST APIAction API
See all▾
MCP ServersGuides & SDKModels
Blog
Introducing Taskade App Factory (November 2026)Best AI Music Theory Tools 2026: 20 Tutors and Ear TrainersWhat Is an AI Employee? Roles, Costs, Risks and How to Hire Your First One (2026)
See all▾
How to Build an AI Support Agent for Your Website (2026)Residual Connections Explained: How ResNet's Skip Connections Made Deep Learning Deep (2026)AI Typography Tools 2026: 14 Font, Pairing and CSS ToolsWhy AI Agents Need an Ontology: The Write Path Nobody Talks About (2026)Browser Agents Explained: How AI Agents Use the Web in 2026Your Business. Your Software. (November 2026)Open Knowledge Format (OKF) Explained (2026): Every Rule in Google's Spec for Agent-Readable KnowledgeHistory of Microsoft Copilot: Clippy to Agents (2026)Is Taste the New Moat? What Still Defends Software When AI Builds Everything (2026)How to Build a Safety and Compliance Tracker Without Code (2026)History of Clay.com: From a Spreadsheet to a $7.1B GTM Platform (2017 to 2026)What Is x402? The HTTP 402 Payment Protocol for AI Agents, Explained (2026)How to Build an AI Lead Qualification Agent (No Code, 2026)LLM Wiki Explained (2026): Karpathy's Pattern, Google's OKF, and When a Folder Beats RAGWhat Is Spec-Driven Development? Tools and Evidence (2026)The Workspace Is the Runtime: What AI Agents Need After the Prompt (2026)Introducing Taskade TSK-1: The System Kernel Behind Every App (2026)Run Your Whole Business in One App with Taskade Genesis (June 2026)
AIAutomationProductivityProject ManagementRemote WorkStartupsKnowledge ManagementCollaborative WorkUpdates
Changelog
New Automation Triggers & Row Totals in Taskade EVE (Nov 16, 2026)Read PDFs in Taskade & File Previews in Chat (Nov 14, 2026)Approve From Activity & Steadier App Kit Copies (Nov 13, 2026)
See all▾
Locked List Alerts & Safer Published Apps (Nov 12, 2026)Automation Setup Checklist & PayPal Signature Checks (Nov 11, 2026)One-Click App Connections & Date Fields Over MCP (Nov 10, 2026)Public Agent Chat Privacy & Cleaner Kit Copies (Nov 9, 2026)
Integrations
Slack AutomationGmail AutomationGoogle Sheets Automation
See all▾
Google Calendar AutomationGoogle Drive AutomationHubSpot AutomationStripe AutomationDiscord AutomationGitHub AutomationNotionYouTube AutomationZapier
© 2026 TASKCADE INC.
PrivacyTermsSecurity
Made withTaskade AIforBuilders
BlogAICatastrophic Forgetting…

Catastrophic Forgetting Explained: Why AI Cannot Learn Continuously Yet, and What the Brain Does Instead (2026)

Why neural networks forget old skills when they learn new ones, from the 1989 addition network to loss of plasticity, and what the brain does instead.

Catastrophic forgetting explained: a broken video recorder that lost its tape, for a guide to why AI cannot learn continuously yet
December 1, 202675 min readTaskade TeamAI#catastrophic-forgetting#continual-learning#lifelong-learning
On this page (53)
Catastrophic Forgetting at a GlanceWhat Is Catastrophic Forgetting?Why Does Catastrophic Forgetting Happen?The History of Catastrophic Forgetting, 1989 to 2026The master timeline1989: the network that forgot the ones facts1990: Ratcliff and recognition memoryThe 1990s fixes1999: French's reviewWhat Does the Brain Do Instead? Complementary Learning SystemsThe evidence for replayThe human version of replayHuman versus machine, side by sideReplay Comes to Machines: Experience Replay and Generative Replay1992: Lin's experience replay2015: the DQN replay buffer2017: generative replay and exemplar memoryElastic Weight Consolidation and the Regularization FamilyWhere the sources disagreeThe rest of the familyGrowing the Network: Progressive Networks and Parameter IsolationThe Method Families ComparedThe Three Continual-Learning ScenariosLoss of Plasticity: The Second ProblemWhat helped and what hurtThe fix: continual backpropagationCatastrophic Forgetting in Large Language ModelsAdapters reduce forgetting and learning togetherHow you train matters as much as what you trainSparse updates forget far lessThe 2025 to 2026 State: Memory Outside the WeightsTest-time training and memory that updates during inferenceContext is state, not learningWhy Can't AI Learn Continuously Yet?The Learning Ladder: Six Levels of Continual LearningLevel 0: replay, the rung AI has climbedLevel 1: selective learning, or how much to updateLevel 2: metacognition, or is the question right?Level 3: intuition, or what is worth a lookLevel 4: instinct, or what matters at allLevel 5: community, or many mindsWhy each level learns from the level below, and why that makes it hardThe 2026 Field Evidence: The CRUX StudyThe Navier-Stokes claim, with its caveatsWhat Builders Can Do TodayThe builder decision tableA decision flowchartRetrieval versus fine-tuning, step by stepTreat memory like a belief, not a logWhere Taskade Fits: State, Not LearningGlossaryFrequently Asked QuestionsSources

In 1989, a small neural network learned to add.

It learned the "ones facts" first: 1+1, 1+2, all the way to 9+1. It got them right. Then the researchers taught it the "twos facts": 2+1, 2+2, up to 9+2. It learned those too, quickly. Then they tested the ones facts again, and the network had lost them. Not slowly. Recall of the ones collapsed as soon as training on the twos began.

The two researchers, Michael McCloskey and Neal Cohen, gave the effect a name that has stuck for 37 years: catastrophic interference. Today most people call it catastrophic forgetting, and in 2026 it sits at the center of the most discussed open problem in AI: why a model that writes code, passes exams and drafts legal briefs still cannot learn from the conversation it had with you yesterday.

This guide tells the story in two parts. The first part is a history with primary sources, from the addition network to the 2024 Nature paper that found a second, deeper failure called loss of plasticity, and on to the 2025 and 2026 methods that keep knowledge outside the weights. The second part is a ladder: six levels of continual learning, from replay up to community, and why each level learns from the one below it. The guide ends with what you can do today when your model cannot learn after deployment.

TL;DR: Catastrophic forgetting is the loss of old skills when a neural network learns new ones, first shown in 1989 when a network learned the twos addition facts and lost the ones. The brain avoids it with a fast hippocampus, a slow neocortex and replay in sleep. AI in 2026 keeps new knowledge outside the weights. Try it free →


Catastrophic Forgetting at a Glance

Catastrophic forgetting is the tendency of an artificial neural network to lose previously learned information, abruptly and almost completely, when it is trained on new information. It is the main obstacle to continual learning, the ability to keep learning from a stream of new data without erasing what came before. The table below is the short version of this guide.

Question Short answer Where to read more
What is it? New training overwrites the shared weights that held old skills What is catastrophic forgetting?
Who found it? McCloskey and Cohen, 1989, and Ratcliff, 1990 The history
How does the brain cope? Fast hippocampus, slow neocortex, replay during sleep Complementary learning systems
What are the main fixes? Replay, regularization such as EWC, and network growth Method families compared
What is the second problem? Loss of plasticity: long training destroys the ability to learn Loss of plasticity
What does AI do in 2026? Keep knowledge outside the weights: retrieval, long context, memory files The 2025 to 2026 state
What should a builder do? Pick by cost, freshness and forgetting risk The builder decision table

If you came here from the Bitter Lesson explainer, this guide is the deep version of its section on why models stop learning. If you want the short definition with a brain diagram, the continual learning wiki page has it.


What Is Catastrophic Forgetting?

Catastrophic forgetting is the abrupt loss of old knowledge in a neural network that is trained on new data without the old data. If a network learns task A and then task B, its performance on task A can drop close to zero, even though nothing was deleted on purpose. McCloskey and Cohen called the forgetting "catastrophic" because it was far worse than the gradual forgetting people show.

The cause is a design choice that is also the source of a neural network's power. A network stores every skill in one shared set of numbers, its weights. Training adjusts those weights to reduce the error on whatever data the network sees right now. If the network sees only task B, the training process has no reason to keep the weights that task A depended on. It moves them wherever task B needs them.

Task A: the ones facts1+1, 1+2 ... 9+1 One shared set of weights Task B: the twos facts2+1, 2+2 ... 9+2 Gradient descent reducesthe error on task B only Task B answers improve Task A answers driftnothing in the loss protects them Catastrophic forgetting
Task A: the ones facts1+1, 1+2 ... 9+1 One shared set of weights Task B: the twos facts2+1, 2+2 ... 9+2 Gradient descent reducesthe error on task B only Task B answers improve Task A answers driftnothing in the loss protects them Catastrophic forgetting
Shared weights are why catastrophic forgetting happens. The weights that answer 1+3 also answer 2+3, and training on the twos alone moves them.

Several terms travel with catastrophic forgetting, and they are easy to mix up. The table separates them.

Term What it means How it differs from catastrophic forgetting
Catastrophic interference The original 1989 name Same effect. Interference names the cause, forgetting names the result
Stability-plasticity dilemma The trade-off between keeping old knowledge and absorbing new knowledge The general trade-off. Forgetting is what happens when plasticity wins
Loss of plasticity The network gradually loses the ability to learn anything new The opposite failure: stability wins, and learning stops
Concept drift The world changes, so old answers become wrong A property of the data, not the network
Model collapse A generative model degrades when it trains on its own outputs A data-quality failure, not interference between tasks
Continual learning The research field that tries to learn a stream of tasks without forgetting The goal. Catastrophic forgetting is its first obstacle

Two other names for the field appear in search results. Lifelong learning and incremental learning mean the same thing as continual learning. Some researchers, Richard Sutton among them, reject the need for any of the three names. "You wouldn't have to say continual learning, because it wouldn't make any sense to talk about learning that wasn't continual," he said in an interview with his Oak Lab co-founder Khurram Javed. "I'm not weird. The field is weird."


Why Does Catastrophic Forgetting Happen?

Catastrophic forgetting happens because a neural network uses distributed, overlapping representations: each memory is spread across many weights, and each weight takes part in many memories. Robert French's 1999 review in Trends in Cognitive Sciences made this the central point. The overlap lets a network generalize, and the same overlap makes it fragile when the training data changes.

French also gave a geometric picture that still helps. Think of every possible setting of a network's weights as a point in a huge space. Training on task A moves the network to a region where task A works. Training on task B alone moves it to a region where task B works, with no pressure to stay near the region for task A. A solution that works for both tasks often exists. Sequential training simply does not look for it.

The fix that works in the original experiments is also the simplest. If you train on A and B together, mixed in the same batches, the network finds the shared solution. McCloskey and Cohen's own conclusion was that interference was catastrophic when learning was sequential, and not when it was concurrent.

Start Step 1 Step 2 Step 3 Step 4 Step 5 0 20 40 60 80 100 Training steps on task B Task A accuracy, schematic percent Schematic: task A accuracy while a network trains on task B Task B data only Task B mixed with task A data
Start Step 1 Step 2 Step 3 Step 4 Step 5 0 20 40 60 80 100 Training steps on task B Task A accuracy, schematic percent Schematic: task A accuracy while a network trains on task B Task B data only Task B mixed with task A data
A schematic, not measured data. It shows the shape McCloskey and Cohen described: sequential training collapses the old task, and concurrent training keeps it.

The same limit shows up in an older memory model. In a Hopfield network, the 1982 model of associative memory, each stored pattern is a valley in an energy landscape. Daniel Amit, Hanoch Gutfreund and Haim Sompolinsky calculated in 1985 that a Hopfield network of N neurons can store about 0.138 N random patterns. Past that point recall does not fade gently. The physicists described a blackout in which all the memories vanish at once. Correlated patterns interfere even earlier.

The neuroscience of memory points the same way. Studies of memory traces, called engrams, find that only a small fraction of neurons in a region join any one memory, and that the fraction stays steady across memories. Sparse, separate codes interfere less. Dense, overlapping codes generalize better. A system that wants both needs two kinds of storage, which is exactly where the brain story goes next.

For the mechanics of how a network adjusts its weights in the first place, the explainer on how LLMs work covers gradient descent and backpropagation from the start.


The History of Catastrophic Forgetting, 1989 to 2026

The history of catastrophic forgetting runs in four eras: the problem was named in the cognitive science of the late 1980s, the brain parallel arrived in 1995, the deep learning revival brought replay and regularization methods from 2013 to 2019, and the era of frozen frontier models from 2020 onward moved knowledge outside the weights. The diagram shows the arc. The table after it is the full record.

1980 to 1999: the problem is named 2013 to 2019: deep learning meets forgetting 2020 to 2026: frozen models, outside memory 1980 Grossberg names thestability-plasticity dilemma 1989 McCloskey and Cohen:the ones facts vanish 1990 Ratcliff: one trialof new learning hurts 1992 Lin: experience replay 1995 Complementary learningsystems and pseudo-rehearsal 1999 French review 2015 DQN replay buffer 2016 Progressive networks 2017 EWC, synaptic intelligence,generative replay 2019 Three scenariosfor continual learning 2020 RAG and test-time training 2021 LoRA adapters 2024 Loss of plasticityin Nature 2025 Titans, RL's Razor, sparsememory fine-tuning, Nested Learning 2026 CRUX field testand the learning ladder
1980 to 1999: the problem is named 2013 to 2019: deep learning meets forgetting 2020 to 2026: frozen models, outside memory 1980 Grossberg names thestability-plasticity dilemma 1989 McCloskey and Cohen:the ones facts vanish 1990 Ratcliff: one trialof new learning hurts 1992 Lin: experience replay 1995 Complementary learningsystems and pseudo-rehearsal 1999 French review 2015 DQN replay buffer 2016 Progressive networks 2017 EWC, synaptic intelligence,generative replay 2019 Three scenariosfor continual learning 2020 RAG and test-time training 2021 LoRA adapters 2024 Loss of plasticityin Nature 2025 Titans, RL's Razor, sparsememory fine-tuning, Nested Learning 2026 CRUX field testand the learning ladder

The master timeline

Year Event Who Why it matters
1959 Retroactive interference with A-B and A-C word lists Barnes and Underwood The human study that McCloskey and Cohen later tried to model
1980 The stability-plasticity dilemma is named Stephen Grossberg, Psychological Review The trade-off that every later method negotiates
1982 Hopfield network for associative memory John Hopfield, PNAS Shared-landscape memory with a hard capacity limit
1985 Capacity of about 0.138 N patterns Amit, Gutfreund and Sompolinsky Past the limit, all memories fail at once
1986 Backpropagation popularized Rumelhart, Hinton and Williams, Nature The learning rule whose updates cause the interference
1989 "Catastrophic interference" in the addition network McCloskey and Cohen The founding demonstration
1989 Replication with a caveat Hetherington and Seidenberg Some memory of the ones remained, and relearning was faster
1990 Recognition-memory models forget after one new trial Roger Ratcliff, Psychological Review Confirmed the effect in a second family of models
1992 Experience replay Long-Ji Lin, Machine Learning Store past experience and reuse it during training
1994 Place cells replay waking activity during sleep Wilson and McNaughton, Science Biological evidence for replay
1995 Complementary learning systems theory McClelland, McNaughton and O'Reilly Why the brain needs a fast and a slow learner
1995 Rehearsal and pseudo-rehearsal Anthony Robins, Connection Science Replay without the original data
1997 Pseudo-recurrent and self-refreshing networks French, and Ans and Rousset Two-network designs inspired by the brain
1999 "Catastrophic forgetting in connectionist networks" Robert French, Trends in Cognitive Sciences The review that framed the field for 15 years
2013 Empirical study of forgetting in modern networks Goodfellow and colleagues Brought the problem into the deep learning era
2015 Deep Q-Network with a replay buffer Mnih and colleagues, Nature Replay became standard practice in deep reinforcement learning
2016 Progressive neural networks, Learning without Forgetting Rusu and colleagues, Li and Hoiem Grow the network, or distill the old model's outputs
2017 Elastic weight consolidation Kirkpatrick and colleagues, PNAS Protect the weights that matter for old tasks
2017 Synaptic intelligence, generative replay, iCaRL, GEM Zenke, Shin, Rebuffi, Lopez-Paz and colleagues The method families take their modern shape
2019 Three scenarios for continual learning van de Ven and Tolias Showed that results depend on whether the task is known
2020 Retrieval-augmented generation, test-time training Lewis and colleagues, Sun and colleagues Knowledge from outside the weights, and learning at inference
2021 Synapses as Bayesian inference Aitchison and colleagues, Nature Neuroscience Each synapse may track its own uncertainty
2021 LoRA low-rank adapters Hu and colleagues Cheap fine-tuning that touches few parameters
2023 Forgetting during continual instruction tuning of LLMs Luo and colleagues Forgetting observed in models from 1 billion to 7 billion parameters
2024 "Loss of plasticity in deep continual learning" Dohare and colleagues, Nature The second failure: networks stop learning at all
2024 "LoRA Learns Less and Forgets Less" Biderman and colleagues, TMLR Adapters trade new learning for less forgetting
2024 Titans: learning to memorize at test time Behrouz, Zhong and Mirrokni A memory module that updates during inference
2025 RL's Razor Shenfeld, Pari and Agrawal On-policy reinforcement learning forgets less than supervised fine-tuning
2025 Sparse memory fine-tuning Lin and colleagues, Meta FAIR Update only the memory slots a new fact activates
2025 Nested Learning and the Hope model Behrouz and colleagues, NeurIPS 2025 Memory modules that update at different speeds
2026 "Why AI systems don't learn and what to do about it" Dupoux, LeCun and Malik A cognitive-science blueprint for autonomous learning
2026 CRUX shadow evaluations Kirgis, Kapoor and 22 co-authors Frontier agents do the engineering of research, not the judgment
2026 The learning ladder JT Yu, Universal Resilience Continual learning as a stack of six levels

1989: the network that forgot the ones facts

McCloskey and Cohen published "Catastrophic Interference in Connectionist Networks: The Sequential Learning Problem" in Psychology of Learning and Motivation, volume 24, in 1989. They ran two experiments with standard backpropagation networks. The first is the famous one. The network learned the 17 ones addition facts to criterion, then the 17 twos addition facts. After even one learning trial on the twos facts, the network could no longer answer the ones facts correctly. The two facts that belong to both sets, 1+2 and 2+1, were badly disrupted in the first trials. The wrong answers the network gave for the ones facts often looked like the output for a different number.

The second experiment tried to model a classic human study. In 1959 Barnes and Underwood had people learn a list of word pairs, A-B, and then a second list with the same first words, A-C. People forgot some of the first list, but gradually. The network, trained the same way, forgot the A-B pairs almost entirely by the time it learned the A-C pairs as well as the human subjects had.

The paper's conclusions are worth reading in full, because they still describe the problem.

McCloskey and Cohen's finding (1989) What it means in 2026 terms
Some interference occurs whenever new learning changes weights that old knowledge uses Any fine-tune of a deployed model risks damage to existing skills
The more new learning there is, the more old knowledge is disrupted Long fine-tunes on narrow data forget more
Interference was catastrophic when learning was sequential, but not concurrent Mixing old data into new training, called replay, works
Changes to hidden units, learning rate, overtraining, weight freezing and targets did not prevent it No single setting fixes the problem

A replication in the same year added a nuance that matters. Phillip Hetherington and Mark Seidenberg reran the addition experiment and reported that learning the twos did not completely destroy every trace of the ones, and that the network relearned the ones faster than it had learned them the first time. Psychologists call this savings. It means the old knowledge was damaged, not deleted, which is why "forgetting" in a neural network is sometimes closer to a scrambled index than to an empty shelf.

1990: Ratcliff and recognition memory

Roger Ratcliff found the same effect in a different kind of model. His 1990 paper in Psychological Review, "Connectionist models of recognition memory: Constraints imposed by learning and forgetting functions," trained backpropagation networks to recognize items they had studied. Well-learned information was catastrophically forgotten in small and large networks alike. Even one training trial with new information caused a significant loss of old information, and the outputs were often a blend of old and new inputs. Adding hidden units did not help. Ratcliff tried extra response nodes that would answer only to old or only to new inputs, and they ended up active for every input.

Ratcliff's result closed off an easy objection. The addition network was not a strange case. Forgetting was a property of how these networks learned.

The 1990s fixes

Researchers spent the next decade on fixes that make representations overlap less, or that bring old data back. Most of them work in small networks, and most of them return in modern form later in this guide.

Fix Idea Representative work Modern descendant
Orthogonal or semi-distributed codes Make memories overlap less in the hidden layer Lewandowsky and Li, 1995, and French, 1991 Sparse memory layers, 2025
Node sharpening Push hidden activations toward a few strong units French, 1991 Sparse activation and mixture-of-experts routing
Novelty rule Train on the part of a new input that differs from what the network expects Kortge, 1990 Surprise-gated memory in Titans, 2024
Pretraining Start from a network that already knows the domain McRae and Hetherington, 1993 Pretrained foundation models
Rehearsal Retrain on stored old items alongside new ones Robins, 1995 Replay buffers and data mixing
Pseudo-rehearsal Feed random inputs through the old network, record its outputs, and replay those pairs Robins, 1995 Generative replay, 2017, and distillation
Two-network designs A fast early network and a slow final-storage network French, 1997, and Ans and Rousset, 1997 Complementary learning systems in AI

Pseudo-rehearsal deserves a closer look, because it solved a practical problem that still matters. Rehearsal needs the old data, and often you do not have it. Anthony Robins's 1995 trick was to feed random inputs through the trained network and record what it answered. Those input-and-output pairs, called pseudo-items, capture the shape of what the network knew. Mixing them into new training protects the old function without storing a single original example. Twenty-two years later, generative replay used the same logic with a neural generator in place of random noise.

1999: French's review

Robert French's "Catastrophic forgetting in connectionist networks," published in Trends in Cognitive Sciences in 1999, summarized the first decade. Its core claim was that the problem comes from the same distributed representations that make networks useful, so any cure must reduce overlap without losing generalization. French pointed to the brain's division of labor between the hippocampus and the neocortex as the most promising model. The review became a standard reference for the field until deep learning brought the problem back at a much larger scale.


What Does the Brain Do Instead? Complementary Learning Systems

The brain avoids catastrophic forgetting by using two learning systems with different speeds, according to the theory that James McClelland, Bruce McNaughton and Randall O'Reilly published in Psychological Review in 1995. The hippocampus learns new episodes quickly with sparse, separate codes. The neocortex learns slowly with overlapping codes that extract general structure. Replay of hippocampal memories teaches the neocortex gradually, interleaved with what it already knows.

The paper's full title states the argument: "Why there are complementary learning systems in the hippocampus and neocortex: Insights from the successes and failures of connectionist models of learning and memory." The authors took catastrophic interference as a clue, not a defect. If a slow, distributed learner like the neocortex were trained on each new experience as it happened, it would suffer the same interference as the addition network. So the brain needs a second system that can store an experience at once, without disturbing the slow one, and then feed it back in small doses.

New experienceduring the day Hippocampusfast, sparse, one-shot Replayreactivation during rest and sleep Interleavingnew memory mixed with old Neocortexslow, overlapping, structured Consolidated knowledgenew skill added, old skills kept
New experienceduring the day Hippocampusfast, sparse, one-shot Replayreactivation during rest and sleep Interleavingnew memory mixed with old Neocortexslow, overlapping, structured Consolidated knowledgenew skill added, old skills kept

The 1995 paper tested the idea in simulation. It used a small semantic network that already knew facts about animals and plants, then taught it a new fact that cut across its categories: a penguin is a bird that swims and does not fly. Taught alone and repeatedly, the new fact disrupted what the network knew about other birds. Taught interleaved with the old facts, it slotted in without the damage. That is the addition network's lesson, with a theory of why the brain would bother.

The evidence for replay

The theory leaned on a finding from the year before. In July 1994, Matthew Wilson and Bruce McNaughton reported in Science that place cells in the rat hippocampus, which fire at particular locations during a maze run, fire again in the same patterns during slow-wave sleep after the run. "Information acquired during active behavior is thus re-expressed in hippocampal circuits during sleep," they wrote, "as postulated by some theories of memory consolidation." Later work found that replay often runs faster than the original experience, at a compressed timescale.

one exposure stores the episode replay during slow-wave sleep, compressed in time small weight change, mixed with old knowledge replay again on later nights next day, the knowledge is available without the episode sparse code, little overlap with old memories general structure extracted, old skills kept Waking experience Hippocampus Neocortex
one exposure stores the episode replay during slow-wave sleep, compressed in time small weight change, mixed with old knowledge replay again on later nights next day, the knowledge is available without the episode sparse code, little overlap with old memories general structure extracted, old skills kept Waking experience Hippocampus Neocortex

The table compares the two systems as the theory describes them.

Property Hippocampus Neocortex
Learning speed Fast, often after one exposure Slow, over many exposures
Code Sparse and pattern-separated Distributed and overlapping
Strength Keeps episodes apart, so they do not interfere Finds structure shared across many episodes
Weakness Limited capacity, poor at generalization Suffers interference if trained on one thing at a time
Role in consolidation Stores the new memory and replays it Absorbs the replayed memory slowly, interleaved with old ones
Machine analogue A replay buffer, a memory file, a retrieval index The weights of the model

In 2016, Dharshan Kumaran, Demis Hassabis and James McClelland updated the theory in Trends in Cognitive Sciences under the title "What learning systems do intelligent agents need?" They tied it directly to machine learning, including the replay buffer in DeepMind's Atari agent, and argued that replay can be weighted toward the experiences that matter most rather than replayed uniformly. The memory consolidation wiki page covers the neuroscience in more depth.

The human version of replay

You use replay every time you review old material while you learn new material. Spaced repetition, the study method that schedules reviews at growing intervals, is the deliberate human version of what the hippocampus does at night: bring the old item back before it fades, mixed in with the new. The spaced repetition explainer covers the method and the forgetting curve behind it, and the guide to studying with AI shows how to schedule reviews with an assistant.

Human versus machine, side by side

Capability Human brain Standard neural network Closest machine method
Learn a new skill without losing an old one Usually, with gradual forgetting No, unless old data returns Replay, regularization
Store an episode after one exposure Yes, in the hippocampus No, needs many gradient steps Replay buffer, retrieval index
Consolidate during rest Yes, replay during sleep No offline phase after release Periodic retraining
Track how sure it is of each belief Proposed at the synapse level One learning rate for most weights EWC, Bayesian methods
Keep the ability to learn for decades Yes, with decline in old age Declines with long training Continual backpropagation
Relearn faster than first learning Yes, savings Yes, savings were seen in 1989 Warm-starting from old weights

In his October 2025 interview with Dwarkesh Patel, Andrej Karpathy put the gap in the brain's own terms. Today's agents, he said, "don't have continual learning. You can't just tell them something and they'll remember it." He compared the missing piece to sleep, when a person's day is distilled back into the brain, and said large language models have no equivalent phase.


Replay Comes to Machines: Experience Replay and Generative Replay

Replay is the oldest and most reliable cure for catastrophic forgetting in machines: store some past experience and mix it into new training, so the network never trains on new data alone. Long-Ji Lin introduced experience replay for reinforcement learning agents in 1992. DeepMind's Deep Q-Network made it standard in 2015. Generative replay in 2017 replaced the stored data with a model that generates it.

1992: Lin's experience replay

Long-Ji Lin, then at Carnegie Mellon University, published "Self-improving reactive agents based on reinforcement learning, planning and teaching" in Machine Learning in 1992. He compared eight learning frameworks and three ways to speed up learning, and experience replay was one of the three. The agent kept a record of its past experiences and presented them to its learning algorithm again, as if they were happening for the first time. Replay made rare and costly experiences count more than once.

2015: the DQN replay buffer

When DeepMind's Deep Q-Network learned to play Atari games from pixels, reported in Nature in February 2015, replay was one of its two key stabilizers. The agent stored its most recent one million transitions in a replay memory and trained on random minibatches drawn from it. Random sampling broke the correlation between consecutive frames, and it meant the network kept seeing old situations while it learned new ones.

Environment New transitionstate, action, reward, next state Replay bufferthe most recent one million transitions Random minibatchold and new experience mixed Network updateno long run of one situation Better policyacts in the environment again
Environment New transitionstate, action, reward, next state Replay bufferthe most recent one million transitions Random minibatchold and new experience mixed Network updateno long run of one situation Better policyacts in the environment again

The reinforcement learning environments guide covers how agents collect the experience that fills buffers like this one.

2017: generative replay and exemplar memory

Storing raw data has two limits. It costs memory, and sometimes you may not keep the data, for privacy or licensing reasons. Hanul Shin, Jung Kwon Lee, Jaehong Kim and Jiwon Kim answered both at NeurIPS 2017 with "Continual Learning with Deep Generative Replay." Their design pairs a generator, a deep generative model, with a solver, the task network. The generator learns to produce samples that look like past tasks, and those samples are interleaved with the new task's data. The authors cited the hippocampus as their inspiration.

The same year produced two influential replay variants. iCaRL, from Sylvestre-Alvise Rebuffi and colleagues at CVPR 2017, kept a small set of real exemplars per class and classified new inputs by their distance to class averages. Gradient Episodic Memory, from David Lopez-Paz and Marc'Aurelio Ranzato at NeurIPS 2017, used a stored memory differently: it allowed only those weight updates that do not increase the loss on old examples. In 2020, Gido van de Ven, Hava Siegelmann and Andreas Tolias published brain-inspired replay in Nature Communications, in which the network replays its own internal representations through feedback connections, closer to what the brain is thought to do.

Replay has one limit no variant escapes, and it explains why frontier labs do not retrain every day. Each new round of learning repeats a large share of the old data. For a model trained on a large slice of the public internet, that is close to the original training cost. JT Yu's summary of this level, discussed below, is blunt: replay is "solved, but expensive."


Elastic Weight Consolidation and the Regularization Family

Elastic weight consolidation, or EWC, protects old skills without replaying old data. It estimates how important each weight was for the old task and adds a penalty that pulls important weights back toward their old values during new training. James Kirkpatrick and 13 colleagues at DeepMind and Imperial College London published it in PNAS in March 2017 under the title "Overcoming catastrophic forgetting in neural networks."

The authors described EWC as an algorithm "analogous to synaptic consolidation" in the brain, a process in which some synapses become less plastic once a memory matters. EWC slows learning on the weights that task A depends on, and leaves the rest free for task B. It measures importance with the Fisher information, a statistic that says how much the model's predictions on task A change when a given weight changes. The paper tested EWC on supervised tasks, including permuted handwritten digits, and on reinforcement learning, where an agent learned several Atari games one after another.

Finish training on task A Estimate each weight's importanceFisher information Attach a spring to each weightstiff if important, loose if not Train on task Bloss = task B error + spring penalty Important weightsbarely move Unimportant weightsmove freely Task A kept, task B learned
Finish training on task A Estimate each weight's importanceFisher information Attach a spring to each weightstiff if important, loose if not Train on task Bloss = task B error + spring penalty Important weightsbarely move Unimportant weightsmove freely Task A kept, task B learned

EWC has a clean Bayesian reading. After task A, the network holds a belief about which weights are right. The best belief after task B combines that old belief with the new evidence. EWC approximates the old belief with a simple bell-shaped curve around each weight, wide where the weight did not matter and narrow where it did. That reading becomes important in the ladder section, because it makes EWC the cheap machine version of a brain that tracks how certain it is.

Where the sources disagree

The EWC paper drew a published correction. In February 2018, Ferenc Huszár wrote in PNAS that the paper's penalty is not the correct one when more than two tasks are learned in sequence, and he derived an alternative. Kirkpatrick and colleagues replied in the same issue that their version was "empirically valid," and argued that it forces the network to remember older tasks more vividly, which can compensate for the fact that older tasks are harder to remember. Both papers are worth reading, and later work uses both forms.

A second disagreement is about scope. EWC works well when the system knows which task it faces. It works poorly when it must work out the task itself, as the next section shows.

The rest of the family

Method Year and venue How it measures importance Note
Elastic weight consolidation 2017, PNAS Fisher information after each task The reference method
Synaptic intelligence 2017, ICML Each weight's contribution to the loss decrease along the training path Computes importance online, during training
Memory aware synapses 2018, ECCV Sensitivity of the output to each weight, no labels needed Works without task labels
Learning without Forgetting 2016, ECCV No weight importance. Distills the old model's outputs on new data A data-level regularizer
Variational continual learning 2018, ICLR A full approximate posterior over weights The more complete Bayesian version, at higher cost

Friedemann Zenke, Ben Poole and Surya Ganguli's synaptic intelligence is the closest relative. Their ICML 2017 paper describes "intelligent synapses" that accumulate task-relevant information over time and use it to learn new information without erasing old knowledge. EWC computes importance once, after a task. Synaptic intelligence computes it continuously, which is closer to how a brain would have to do it.


Growing the Network: Progressive Networks and Parameter Isolation

The third way to stop forgetting is to stop sharing: give each new task its own parameters, so new learning cannot touch old ones. Progressive neural networks, from Andrei Rusu and colleagues at DeepMind in June 2016, add a new column of layers for each task, freeze the old columns, and connect the new column to the old ones with lateral connections so it can reuse their features.

Progressive networks cannot forget, by construction. The authors tested them on Atari and 3D maze games and found that they transferred knowledge better than baselines based on pretraining and fine-tuning. The cost is growth: the network gets bigger with every task, and something must tell it which column to use.

Two other designs isolate parameters without unlimited growth. PackNet, from Arun Mallya and Svetlana Lazebnik at CVPR 2018, prunes the weights a task does not need after training and frees them for the next task. Mixture-of-experts routing, common in large language models, sends each input to a small subset of expert networks. Routing is not a continual-learning method in itself, but it uses the same principle of separation that the hippocampus uses: keep different things in different places, and they interfere less. Meta's 2025 sparse memory fine-tuning, covered later, is the newest version of that idea.


The Method Families Compared

Continual-learning methods fall into four families: replay, which brings old data back; regularization, which protects important weights; architecture, which gives new tasks new parameters; and memory outside the weights, which leaves the model unchanged and supplies knowledge at run time. The first three change weights. The fourth is the main approach of 2026, and by a strict definition it is not learning at all.

ContinualLearningMethod + keep old skills: goal + learn new ones: goal Replay + 1995: rehearsal + replay 1992: experience + replay 2017: generative + and GEM 2017: iCaRL Regularization + 2017: EWC + intelligence 2017: synaptic + aware synapses 2018: memory + without forgetting 2016: learning Architecture + networks 2016: progressive + 2018: PackNet + adapters 2021: LoRA + memory fine-tuning 2025: sparse OutsideTheWeights + 2020: retrieval + context: long + memory files: agent + stay frozen: weights
ContinualLearningMethod + keep old skills: goal + learn new ones: goal Replay + 1995: rehearsal + replay 1992: experience + replay 2017: generative + and GEM 2017: iCaRL Regularization + 2017: EWC + intelligence 2017: synaptic + aware synapses 2018: memory + without forgetting 2016: learning Architecture + networks 2016: progressive + 2018: PackNet + adapters 2021: LoRA + memory fine-tuning 2025: sparse OutsideTheWeights + 2020: retrieval + context: long + memory files: agent + stay frozen: weights
Family How it prevents forgetting Strength Weakness Best fit
Replay Mixes old data, or generated stand-ins, into new training The most reliable family across scenarios Repeats old training cost, needs data or a generator When you can keep data and pay for retraining
Regularization Penalizes change to weights that mattered No stored data needed Fails when the task is not known at test time Task-labeled settings, small models
Architecture Gives new tasks new or separate parameters No forgetting of isolated parts Growth, routing and task identity problems Many distinct tasks, adapters per customer
Outside the weights Leaves the model unchanged, supplies knowledge at run time Zero forgetting of model skills, instant updates The model does not improve, context limits apply Deployed products in 2026

The four families also differ on two axes a builder cares about: what the method costs to keep running, and how well it keeps old skills. Mermaid's quadrant charts do not render on this site, so the diagram below draws the four quadrants as boxes.

Low cost, high retention High cost, high retention Low cost, low retention Medium cost, medium retention Retrieval and memory filesweights untouched Adapters per taskbase model frozen Full replay retrainingold data mixed back in Progressive networksa column per task Naive fine-tuningnew data only EWC and synaptic intelligenceweight penalties Generative replaya second model to train
Low cost, high retention High cost, high retention Low cost, low retention Medium cost, medium retention Retrieval and memory filesweights untouched Adapters per taskbase model frozen Full replay retrainingold data mixed back in Progressive networksa column per task Naive fine-tuningnew data only EWC and synaptic intelligenceweight penalties Generative replaya second model to train
A qualitative map, not a benchmark. "Retention" here means retention of the base model's existing skills. Retrieval keeps them perfectly because it changes no weight, but it also teaches the model nothing.

The Three Continual-Learning Scenarios

Whether a continual-learning method works depends on what the model must figure out at test time, according to Gido van de Ven and Andreas Tolias. Their 2019 paper "Three scenarios for continual learning" defined task-incremental, domain-incremental and class-incremental learning, and showed that methods that look strong in the easiest scenario can fail in the hardest. A 2022 follow-up in Nature Machine Intelligence, with Tinne Tuytelaars, called them "three types of incremental learning."

Scenario What the model is told at test time Example What works
Task-incremental Which task the input belongs to "Is this a cat or a dog?" then "Is this a car or a truck?", with the task named Most methods, including EWC and separate output heads
Domain-incremental Nothing, but the task structure stays the same Recognize handwritten digits as the handwriting style changes Replay, some regularization
Class-incremental Nothing, and it must tell apart classes from all past tasks Learn cats and dogs, then cars and trucks, then tell a cat from a truck Replay is needed. Regularization methods such as EWC fail
yes no no, same output space yes At test time, is thetask identity given? Task-incrementalthe easiest case Must the model also inferwhich task the input is from? Domain-incrementalthe input distribution shifts Class-incrementalthe hardest case Most methods work Replay and someregularization work Replay appears necessary
yes no no, same output space yes At test time, is thetask identity given? Task-incrementalthe easiest case Must the model also inferwhich task the input is from? Domain-incrementalthe input distribution shifts Class-incrementalthe hardest case Most methods work Replay and someregularization work Replay appears necessary

The headline result in the 2019 abstract is plain: when task identity must be inferred, "regularization-based approaches (e.g., elastic weight consolidation) fail," and the authors conclude that replaying representations of earlier experiences appears necessary to solve the scenario. That finding explains a decade of confusing benchmark results. Many papers reported strong numbers in the task-incremental setting, where the model is handed a large hint, and those numbers did not carry over to the setting a real deployed system faces.

The real world is closer to the hardest scenario. A deployed assistant is never told "this question belongs to task 47." That is one reason the field moved toward replay, and then, for frontier models, toward keeping new knowledge outside the weights entirely. The history of how AI evaluation handles shifting tests is its own story, covered in part by the agent memory negative results guide.


Loss of Plasticity: The Second Problem

Loss of plasticity is a deeper failure than catastrophic forgetting: a network trained on a long stream of tasks gradually loses the ability to learn new ones at all. Shibhansh Dohare, J. Fernando Hernandez-Garcia, Qingfeng Lan, Parash Rahman, A. Rupam Mahmood and Richard Sutton of the University of Alberta reported it in Nature on August 21, 2024, in "Loss of plasticity in deep continual learning" (volume 632, pages 768 to 774).

Forgetting means old knowledge leaks out. Loss of plasticity means new knowledge cannot get in. A network can suffer both, and the cures pull in opposite directions: protecting old weights reduces forgetting and can make plasticity loss worse.

The paper's clearest experiment is Continual ImageNet. Each task asks the network to tell two image classes apart, and each new task uses a new pair, for 2,000 tasks in a row. The network was initialized once, at the start. In the preprint's words, binary classification performance "dropped from 89% accuracy on an early task down to 77%, about the level of a linear network, on the 2000th task." The published version reports up to 88 percent on early tasks. Either way, after enough tasks the deep network learned new tasks no better than a model with no hidden layers.

Early task Task 2000 50 60 70 80 90 100 Test accuracy percent Continual ImageNet: accuracy on a new task, early versus late
Early task Task 2000 50 60 70 80 90 100 Test accuracy percent Continual ImageNet: accuracy on a new task, early versus late
The two endpoints reported in the Dohare et al. preprint (arXiv:2306.13812). The 77 percent figure is close to what a linear network achieves on the same tasks. The published Nature version reports up to 88 percent on early tasks.

The effect appeared in reinforcement learning too. A simulated ant robot trained with a standard algorithm, PPO, performed well until the friction of its environment changed, then declined. In a version with constant friction, its average reward rose for about 3 million steps and then collapsed, and by 20 million steps the ant failed every episode.

What helped and what hurt

Technique Effect on plasticity in the 2024 paper
Standard Adam optimizer settings Severe loss of plasticity
Adam with tuned settings, both decay rates at 0.99 Much better in reinforcement learning
Dropout Made performance worse, and more dropout made it worse still
Online normalization Better early, worse than plain backpropagation later
L2 regularization Reduced the loss, but sensitive to its settings
Shrink and Perturb, shrinking weights and adding small noise Nearly eliminated the loss on one benchmark, sensitive to the noise scale
Continual backpropagation Maintained plasticity across the tested settings

One row conflicts with an earlier result, and the conflict is instructive. In 2013, Ian Goodfellow and colleagues found that dropout was consistently the best choice for helping a network adapt to a new task while it remembered the old one. In 2024, dropout made loss of plasticity worse. The two studies measured different things, over very different numbers of tasks. A setting that reduces forgetting across two tasks can damage learning across two thousand.

The fix: continual backpropagation

Standard backpropagation uses randomness only once, when it initializes the weights. As training goes on, that variety gets used up: units go dormant, weights grow large, and the network's features become less diverse. Continual backpropagation keeps injecting variety. At each step, it reinitializes a very small fraction of the least useful hidden units, measured by a running average of each unit's activation times the size of its outgoing weights. New units start with zero outgoing weights, so they do not disturb the network's output, and a maturity threshold protects them for a while.

random initialization gives diverse features many tasks later, units go dormant new tasks learned no better than a linear model continual backprop reinitializes low-utility units fresh variety, old useful units kept Fresh Learning Saturated Stuck Refreshed Adapting
random initialization gives diverse features many tasks later, units go dormant new tasks learned no better than a linear model continual backprop reinitializes low-utility units fresh variety, old useful units kept Fresh Learning Saturated Stuck Refreshed Adapting

Sutton describes the idea as generate-and-test inside the network. "Backprop only has random weights at the beginning of time," he said in the same interview with Javed. "And then as you go on, all that variety from the randomness gets used up. With continual backprop, we keep injecting a bit of randomness, a bit of generate-and-test, and the operation of backprop is the tester." The Bitter Lesson explainer covers his second prescription, a separate learned step size for every weight.

Javed adds a limit that matters for anyone who hopes to bolt this onto an existing model. These algorithms, he said, "meta-learn how to learn. So really you have to say, I'm going to learn from scratch." They are a path for new systems, not a patch for frozen ones.


Catastrophic Forgetting in Large Language Models

Large language models forget like any other neural network when you change their weights. Fine-tuning a model on a narrow dataset can erode its general knowledge, reasoning and writing quality, and continual fine-tuning on a sequence of datasets compounds the damage. This is a main reason frontier models ship with frozen weights and receive new knowledge through retrieval, long context and memory files instead. The guide to how LLMs work explains the weights that fine-tuning changes.

In August 2023, Yun Luo and colleagues at Westlake University and Tencent published "An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuning." They tuned models on a sequence of instruction datasets and measured domain knowledge, reasoning and reading comprehension along the way. Catastrophic forgetting was "generally observed" in models from 1 billion to 7 billion parameters. They also found that general instruction tuning before the later fine-tuning reduced the forgetting, a modern echo of the 1993 finding that pretraining reduces interference.

Adapters reduce forgetting and learning together

LoRA, introduced by Edward Hu and colleagues in 2021, fine-tunes a model by training small low-rank matrices beside the frozen weights. Because it changes so little, people hoped it would forget little. Dan Biderman and colleagues at Columbia University and Databricks tested that hope in "LoRA Learns Less and Forgets Less," published in Transactions on Machine Learning Research in 2024. They fine-tuned a 7-billion-parameter model on code and on math. LoRA kept more of the base model's general skills than full fine-tuning did, and it also learned less of the new domain. The paper notes that full fine-tuning learns weight changes with a rank 10 to 100 times higher than typical LoRA settings.

Researchers still study why. A 2024 follow-up, "LoRA vs Full Fine-tuning: An Illusion of Equivalence," confirmed that LoRA forgets less than full fine-tuning. It traced most of LoRA's forgetting to new high-ranking directions in the weights, which the authors call intruder dimensions, and warned that these can add up over sequential fine-tuning. The safe reading is that LoRA is a strong default for limiting damage, not a guarantee.

How you train matters as much as what you train

In September 2025, Idan Shenfeld, Jyothish Pari and Pulkit Agrawal of MIT published "RL's Razor: Why Online Reinforcement Learning Forgets Less," accepted at ICLR 2026. They compared supervised fine-tuning with on-policy reinforcement learning at equal performance on a new task, and found that reinforcement learning preserved prior skills significantly better. Their explanation: among all the ways to solve a new task, on-policy reinforcement learning tends to pick the one closest to the original model, and forgetting tracked how far the new model drifted from the base model.

Sparse updates forget far less

In October 2025, Jessy Lin and colleagues at Meta FAIR published "Continual Learning via Sparse Memory Finetuning." Their model has memory layers, large tables of slots that only a few inputs use at a time. To learn a new fact, the method updates only the slots that the fact activates strongly compared with how often pretraining data uses them. On a held-out benchmark, the measured forgetting was stark.

Full fine-tuning LoRA Sparse memory fine-tuning 0 20 40 60 80 100 Drop in held-out performance percent Forgetting on a held-out benchmark after learning new facts
Full fine-tuning LoRA Sparse memory fine-tuning 0 20 40 60 80 100 Drop in held-out performance percent Forgetting on a held-out benchmark after learning new facts
Figures from Lin et al., "Continual Learning via Sparse Memory Finetuning" (arXiv:2510.15103, October 2025), at a comparable level of learning on the new facts.

The result connects directly to 1990s ideas. Sparse, separate codes interfere less, which is why French pushed for semi-distributed representations in 1991 and why the hippocampus uses sparse codes. Thirty-four years later, sparsity is again the lead candidate.


The 2025 to 2026 State: Memory Outside the Weights

The dominant answer to catastrophic forgetting in 2026 is to avoid weight changes after release. Frontier models are trained, evaluated and frozen. New knowledge reaches them through retrieval, long context windows and memory files that agents read and write. Because the weights never change, these methods cannot overwrite an old skill. By a strict definition, they also do not learn.

Each of these methods has its own history on this blog. The history of retrieval-augmented generation traces RAG from the 2020 paper by Patrick Lewis and colleagues to agentic search. The context window history covers how windows grew large enough to hold whole codebases. The agent memory history follows memory files and persistent memory for agents, and the types of AI agent memory guide sorts them into episodic, semantic and working memory.

Approach Example Do the weights change? Forgetting risk for model skills Main limit
Retrieval Search an index of documents and paste the best matches into the prompt No None Recall quality, and the model does not get better at the task
Long context Put the whole codebase or contract in the window No None Cost per call, and recall degrades in very long contexts
Agent memory files An agent writes notes and reads them in later sessions No None for the model, but a bad write can overwrite a good note Someone must decide what to keep
Fine-tuning and adapters Train an adapter on your support tickets Yes Moderate Forgetting, evaluation, upgrade cost
Test-time training Update a memory module during inference Yes, a small part Under study Early research, mostly at small scale
Periodic retraining A lab refreshes the model with new data mixed with old Yes Controlled by replay Cost, slow cadence, aggregate only

Test-time training and memory that updates during inference

A research line that started in 2020 tries to put a small amount of learning back into inference. Yu Sun, Xiaolong Wang, Zhuang Liu, John Miller, Alexei Efros and Moritz Hardt introduced test-time training at ICML 2020: before the model answers, it takes a few self-supervised training steps on the test input itself. On December 31, 2024, Ali Behrouz, Peilin Zhong and Vahab Mirrokni of Google Research posted Titans: Learning to Memorize at Test Time, later published at NeurIPS 2025. Titans adds a neural long-term memory module whose own parameters change during inference according to a "surprise" signal, while attention handles the short-term context.

In November 2025, Google Research introduced Nested Learning, published at NeurIPS 2025, with a model called Hope. Its continuum memory system is a chain of blocks that update at different frequencies, so memories are stored at different timescales, from fast to slow. That is a mechanical echo of the hippocampus and neocortex. Reaction was mixed. Bing Liu of the University of Illinois Chicago, author of a textbook on lifelong learning, told Transformer in January 2026 that the approach is "probably just making the model forget a little bit less."

Context is state, not learning

Richard Sutton draws the line that matters for everything in this section. Asked about memory features and in-context learning in his interview with Khurram Javed, he answered in four words: "Their weights never change." Then he explained: "You can change the state of the model by giving it more context... if the state says something new, then it will use that to make the next prediction. But the model is not learning."

Sutton does count one case as real continual learning: a deployed model, such as a code-completion tool, that is retrained on usage pooled across many users. Javed named its limit. "I don't want to fight with 100,000 other people about what they want to teach their models," he said. "I want to teach my model something very specific, and I want to do it to my version of the model." That is the per-user stream problem, and in 2026 no frontier lab ships a solution to it.

evaluation and safety testing weights locked each conversation adds context the conversation ends, weights unchanged the lab retrains on pooled data, months later new evaluation, then a new frozen model Trained Frozen Deployed ContextState SessionEnds Refreshed NewRelease
evaluation and safety testing weights locked each conversation adds context the conversation ends, weights unchanged the lab retrains on pooled data, months later new evaluation, then a new frozen model Trained Frozen Deployed ContextState SessionEnds Refreshed NewRelease

In March 2026, Emmanuel Dupoux, Yann LeCun and Jitendra Malik published "Why AI systems don't learn and what to do about it: Lessons on autonomous learning from cognitive science." They propose three parts: a System A that learns from passive observation, a System B that learns from the system's own actions, and a System M, a meta-control layer that switches between them based on internally generated signals. System M is a metacognitive controller, and it is the bridge to the second half of this guide.


Why Can't AI Learn Continuously Yet?

AI cannot learn continuously yet because five problems stack on top of each other: weight updates overwrite old skills, long training destroys the ability to learn, a model that changes after release is hard to test and keep safe, per-user learning is expensive, and nobody has a reliable way to decide what is worth learning. The first two are algorithmic. The next two are practical. The last one is the hardest, and it is where the learning ladder begins.

Problem What goes wrong Status in 2026
Catastrophic forgetting New updates overwrite old skills in shared weights Reduced by replay, adapters and sparsity, not solved
Loss of plasticity Long training on a stream makes new learning fail Fixes such as continual backprop work in research settings
Safety and testing A model that changes after testing is no longer the tested model, and bad inputs can poison it The main reason labs freeze weights
Per-user cost A separate learning model for each user multiplies training and serving cost Adapters lower the cost, pooled retraining avoids it
Judgment The system must decide what to learn, how much to change, and when its frame is wrong Largely unsolved. People supply it

The safety point deserves a sentence of its own. Sean Goedecke, an engineer who writes about AI, argued in February 2026 that hooking a model up to learn from its inputs is mechanically easy, and that the hard part is "changing the model in ways that make it better, not worse." A model that learns from what users send it can also learn whatever an attacker sends it. Yejin Choi, speaking at Davos, called mixing training and testing "almost a sin," because post-deployment learning can void the safety tests run before release.

The gap between what a model can produce and what it can judge is also the thread of the metacognitive AI guide and of the research on self-improving agents that reflect on their work. The ladder below puts that gap in order.


The Learning Ladder: Six Levels of Continual Learning

The learning ladder is a framework from the video "AI Has Finally Hit the Brain's Hardest Problem," published on November 2, 2026, by JT Yu on the Universal Resilience channel (watch on YouTube). Its argument is that continual learning is not one skill but a stack of six, and that each level learns from the level below. Replay is the bottom rung, and it is the only one AI has solved.

The video calls continual learning "the biggest obstacle to AGI." The idea of the ladder belongs to the video. The neuroscience and machine-learning sources attached to each level below are this guide's additions, chosen to show where each level stands in the research.

Level 0: Replaydo not overwrite the old Level 1: Selective learninghow much should I update? Level 2: Metacognitionis my question right? Level 3: Intuitionwhich options are worth a look? Level 4: Instinctwhat matters at all? Level 5: Communitywho catches what one mind misses?
Level 0: Replaydo not overwrite the old Level 1: Selective learninghow much should I update? Level 2: Metacognitionis my question right? Level 3: Intuitionwhich options are worth a look? Level 4: Instinctwhat matters at all? Level 5: Communitywho catches what one mind misses?
Level The question it answers Human mechanism, per the video State of AI, per the video The video's example
0. Replay How do I keep old skills while I learn new ones? Practice the old skill beside the new one Solved, but expensive The 1989 addition network
1. Selective learning Is this worth learning, and how much should I change? The brain tracks how certain each belief is The full Bayesian version costs too much at scale A closed coffee shop versus gravity switched off
2. Metacognition Is my set of explanations wrong? Is this the right question? Prefrontal circuits that judge the work Reflection uses largely the same system that did the work Le Verrier's Vulcan versus Einstein
3. Intuition Which options are worth a look at all? Years of experience compressed into templates Works in closed games, not in the open world Chess masters, AlphaGo
4. Instinct What matters in the first place? Pain, joy, hunger, fear and curiosity, tuned by evolution No account of how to train it A wedding remembered, a lunch forgotten
5. Community Who catches the mistakes no individual catches? Science, institutions and cultures People around the model do this job today "The community itself is another level of learning"

Level 0: replay, the rung AI has climbed

Replay is everything in the first half of this guide. Mix old examples into new training and the old skill survives. Modern pretraining samples many domains together for the same reason. The video's verdict is that this level is solved, and that the limit is cost: each new round of learning repeats the full mix, so a lab cannot ship a newly pretrained model every day.

The video also names the workaround the industry uses, and gives it a cool reception. Web search and other lookups supply new facts without retraining, but "looking it up doesn't work nearly as well as actually learning it." That is Sutton's point about state, said in plain words.

Level 1: selective learning, or how much to update

Selective learning is the ability to decide how much a new piece of evidence should change a belief. The video's example: one person can change your belief that a coffee shop is open, but a hundred people cannot change your belief in gravity. "Your brain doesn't just store what you know," the video says. "It also keeps track of how certain you are about what you know."

Evidence has quality, not only quantity. A hundred people who all watched the same viral video are one source, not a hundred. Universities with different instruments are many sources. That is the logic of Bayesian inference: weigh the certainty of the current belief against the strength of the new evidence.

low certaintythe coffee shop is open high certaintygravity works no, one sourcerepeated many times yes, many independentinstruments agree New evidence arrives How certain isthe current belief? Small evidencemoves it a lot Is the new evidenceindependent and strong? Belief barely moves Belief updates,carefully
low certaintythe coffee shop is open high certaintygravity works no, one sourcerepeated many times yes, many independentinstruments agree New evidence arrives How certain isthe current belief? Small evidencemoves it a lot Is the new evidenceindependent and strong? Belief barely moves Belief updates,carefully

The video notes that some neuroscientists propose each synapse tracks its own uncertainty, so less certain synapses change more. The paper behind that proposal is Laurence Aitchison, Jannes Jegminat, Jorge Aurelio Menendez, Jean-Pascal Pfister, Alexandre Pouget and Peter Latham, "Synaptic plasticity as Bayesian inference," in Nature Neuroscience in March 2021. They argue that synapses represent a probability distribution over their own strength, not a single value, and that synapses may signal their uncertainty through the variability of their responses. In their account, more uncertain weights get higher learning rates.

The machine version exists, and it is already in this guide. EWC is a cheap approximation of exactly this idea: one certainty value per weight, measured once per task. Variational continual learning, from Cuong Nguyen and colleagues at ICLR 2018, keeps a fuller distribution over weights at a much higher cost. In the video's words, "the full version is far too expensive at scale, and the cheap versions don't work nearly as well."

Sutton and Javed's prescription sits on this rung. "Every weight in your network has to have a separate step size," Javed said. "Most of your network will have weights that have tiny step sizes. So when you train on a new example, they don't get destroyed."

Level 2: metacognition, or is the question right?

Metacognition is thinking about your own thinking: judging how good your work is, deciding what to do next, and asking whether you are working on the right question at all. The video's example is the best story in the history of physics for this rung.

In 1859, the French astronomer Urbain Le Verrier found that the orbit of Mercury did not match Newton's theory. Its closest point to the Sun advanced a little faster than predicted, by about 38 arcseconds per century in his analysis (Simon Newcomb re-estimated it at 43 in 1882). Le Verrier had used the same reasoning to predict Neptune, found in 1846 from its pull on Uranus. So he kept Newton and proposed a hidden planet inside Mercury's orbit. He named it Vulcan. Nobody ever found it reliably.

In 1915, Albert Einstein changed the frame. In general relativity, space and time are not a fixed grid, and the theory predicts Mercury's extra 43 arcseconds per century with no hidden planet. The video draws the lesson: "Bayesian inference can tell you how likely each explanation is, given the evidence. But it can't tell you which explanations to consider in the first place."

Observation: Mercury's orbitdoes not match Newton Update inside the frameBayesian reasoning Question the framemetacognition Keep Newton, add a hidden planetVulcan, never found Change the theory of space and timegeneral relativity, 1915 The search goes on for decades 43 arcseconds per centuryexplained, no new planet
Observation: Mercury's orbitdoes not match Newton Update inside the frameBayesian reasoning Question the framemetacognition Keep Newton, add a hidden planetVulcan, never found Change the theory of space and timegeneral relativity, 1915 The search goes on for decades 43 arcseconds per centuryexplained, no new planet

The video frames human metacognition as the job of the prefrontal cortex, "a part of the brain that evolved to take on different tasks from the rest," and notes that people with damage there "can still perform a task normally, yet lose the ability to tell how well they did." The research is more careful than a single module. Steve Fleming, a cognitive neuroscientist who studies metacognition, describes specific circuits in the prefrontal cortex that contribute to confidence judgments, measured by how well a person's confidence tracks their actual performance. Damage to those networks can impair insight. That is a contribution, not one clean metacognition center. The metacognition guide and the metacognition wiki page cover the science.

The machine contrast in the video is that "when AI reflects on its own work, it's largely using the same system that produced the work, with the same perspectives and assumptions." Research supports a version of this. Jie Huang and colleagues reported in "Large Language Models Cannot Self-Correct Reasoning Yet," at ICLR 2024, that models struggled to self-correct without outside feedback, and that self-correction at times made answers worse. It helps to split metacognition into three jobs, because the machine results differ by job.

Metacognitive job The question Machine form Machine state in 2026
Monitoring How good is this work? Confidence scores, calibration, self-checks Partial: fair on narrow questions
Control What next, and how much effort? Test-time compute, stopping, restarting Weak on open-ended work
Frame revision Is this the right question? No standard benchmark Absent in the 2026 field test

Level 3: intuition, or what is worth a look

Intuition is a lifetime of experience compressed into shortcuts that tell you which options deserve attention. In 1973, William Chase and Herbert Simon published "Perception in chess" in Cognitive Psychology. Players from master to novice looked at a chess position for five seconds and rebuilt it from memory. On positions from real games, recall rose with playing strength. On random positions, the masters' advantage nearly vanished. Masters did not have better memories in general. They had a store of familiar patterns, called chunks, that let them see a real position as a few meaningful groups. A later study by Fernand Gobet and Simon in 1996 found that a small advantage for masters remains even on random positions.

Earlier, Adriaan de Groot's study of chess thought, published in Dutch in 1946 and in English in 1965, found that strong players did not search more moves than weaker players. In the video's words, "The bad moves never even come to mind." The video's definition: "That's what intuition is: a lifetime of experience, compressed into shortcuts that tell you what's worth doing."

The video pairs intuition with metacognition: "You follow a hunch, it turns out wrong, you think about why, and your next hunch gets better." That is a learning loop across two levels, and it is the reason intuition sits above metacognition on the ladder. Intuition is trained by the judgments the level below produces.

AlphaGo is the machine case. DeepMind's 2016 system, reported in Nature, combined a learned policy network, which suggested promising moves, and a value network, which judged positions, with a tree search. The policy network is machine intuition: it keeps the search away from bad moves. AlphaGo beat Lee Sedol four games to one in March 2016. It worked because Go has fixed rules and a clear result, so the system could play itself millions of times. The real world has no fixed rules, no clear win, and often a delay of months or years before you learn whether a move was good. Learning the rules of an open world, instead of being handed them, is the open research problem.

Level 4: instinct, or what matters at all

Instinct is the signal that tells a brain a moment matters before any reasoning starts: pain, joy, hunger, fear, curiosity. The video's example is memory itself. You remember your wedding and forget a lunch from 47 days ago, because emotion marked one as important. Evolution tuned these signals over hundreds of millions of years.

In machine terms, instinct is closest to a reward function: the definition of what counts as success. Reinforcement learning assumes someone supplies it. The video's verdict is that AI has no account of how to train this level. The honest 2026 position is that people supply the goals.

Level 5: community, or many minds

The top rung is the community: science, institutions, markets, cultures. A single scientist can be wrong for a career. A field with independent labs, replication and peer review catches errors no individual catches. "The community itself is another level of learning," the video says. Today, it argues, the people around a model, its builders, reviewers and users, do this job.

Why each level learns from the level below, and why that makes it hard

Each level learns from the outputs of the level below. Selective learning needs stored beliefs that replay keeps intact. Metacognition needs certainty signals from selective learning, so it can see where its beliefs are weak. Intuition is trained by many rounds of metacognitive correction. Instinct sets the reward that tells all of them what matters. The community learns from many individuals.

That dependency is what makes continual learning hard. The video offers an illustration, and says it is an illustration: if each level needs 10 examples from the level below, three levels need a thousand times the experience of the first. Most of that experience cannot be replayed at high speed, because the real world does not run faster than real time.

The video also does the arithmetic for brute force. Choose a research problem, a hypothesis, an experiment, the data, the analysis and the conclusion, six choices with 100 options each, and you get 100 to the sixth power, a trillion possible papers. At one hour per paper on 10,000 agents running in parallel, the search takes more than 11,000 years. An intelligence "has to know what not to try."

One more reading helps tie the ladder to the first half of this guide. Each level sets how fast the level below can change, which makes the ladder a ladder of learning rates. This reading is this guide's, not the video's.

Level What it controls in the level below
Replay One rate for all weights. Old data holds the weights in place
Selective learning A rate for each belief. Certain beliefs move slowly
Metacognition A rate for the set of hypotheses. It decides when the frame must change
Intuition Which options enter the set at all
Instinct The reward that says an outcome matters
Community Which individual conclusions the group keeps

The video's closing line says the same thing in plain words. At every level, learning is a balance between "preserving what's working and changing what's not."


The 2026 Field Evidence: The CRUX Study

The CRUX study is the clearest 2026 field test of whether AI agents can do the upper rungs of the ladder. Its title is "Can AI agents conduct open-ended AI research? Early evidence from two case studies," by Peter Kirgis, Sayash Kapoor and 22 co-authors, posted to arXiv on July 29, 2026 (arXiv:2607.27191). The finding: frontier agents "do the engineering of AI research" but struggle with its critical parts.

The method is called a shadow evaluation. Researchers took the central question of two unpublished, high-quality papers, both NeurIPS 2026 submissions, and gave it to frontier AI agents. The original authors then graded what the agents produced. The authors list starts with Princeton and includes ten more institutions, among them the UK AI Security Institute and Stanford.

Setup detail Value
Research questions The central questions of two unpublished NeurIPS 2026 submissions
Time A 120-hour deadline, and both runs received a 24-hour extension
Budget $3,000 of API credits per run, plus GPU credits
Actual API spend $1,130 and $1,235. Both runs left more than half the budget unused
Agents A frontier model on an agent scaffold, with a second model and scaffold as a robustness check
Review support A self-review subagent and three external AI review tools
Outcome Complete papers with no human help on the engineering. The original authors rejected both
Run 1 Run 2 0 500 1000 1500 2000 2500 3000 US dollars CRUX: API budget versus actual spend per run, in US dollars Budget Spent
Run 1 Run 2 0 500 1000 1500 2000 2500 3000 US dollars CRUX: API budget versus actual spend per run, in US dollars Budget Spent
Both agents could monitor their own API usage in real time and left more than half of the budget unused. Source: Kirgis et al., arXiv:2607.27191.

The paper lists five failure modes. The video names three. The other two are the purest metacognitive failures in the study, because the information the agents needed was in their own context.

CRUX failure mode The paper's description Metacognitive job that failed
Judgment Poor judgment about the bar for publishable research Monitoring
Creative problem solving Uncreative responses to shortcomings in the research design Frame revision
Backtracking Ineffective backtracking from dead ends Control
Resource awareness Poor awareness of the time and budget that remained Monitoring of the agent's own state
Instruction drift Gradual drift from explicit instructions Monitoring of the agent's own goal

The video's own summary is the line most worth keeping: the agents "could produce an enormous amount of work, but they struggled to decide what work was worth doing." The agents sometimes had good ideas and often could not tell which ones.

Two findings sharpen the ladder's claims. First, the study answers the obvious objection that more compute would have fixed it. The authors judged that more time "would have produced a longer paper with the same central limitations," and the spend figures show the agents did not use the budget they had. The limit was the decision, not the supply. That is the authors' judgment, not a tested result.

Second, a separate reviewer did not solve the problem. The agents had a self-review subagent and three external AI reviewers, and they gave a lenient review too much weight. That is the video's own level 1 rule turned on level 2: a reviewer that reads the same draft with similar biases is one source, not two. A monitor needs different evidence, not only a different context.

The Navier-Stokes claim, with its caveats

The video also cites OpenAI's announcement of September 8, 2026, about the Navier-Stokes equations. Press reports describe it as a claimed result for the forced case, blow-up with a smooth external force, and it awaits verification. OpenAI has said it does not intend to claim the Millennium Prize. OpenAI released a Lean formalization with the claim. A proof checker confirms that each step follows from the stated assumptions, not that the formal statement is the famous problem. Credit is also publicly disputed: the mathematicians Levent Alpöge and Tristan Buckmaster posted related work on the forced Euler equations a day earlier, and OpenAI says its agents did not see their unpublished work.

Whatever the final verdict, the case fits the ladder. People chose the problem decades ago. The agents did the middle part. A proof checker checks the steps, and human mathematicians still give the final verdict. The AI research assistants guide covers what research agents do well today, which is that middle part.


What Builders Can Do Today

When your model cannot learn after deployment, you have six practical options: retrieval, long context, memory files, fine-tuning, adapters and periodic retraining. The first three change no weight and cannot cause catastrophic forgetting. The last three change weights and carry forgetting risk in exchange for skills the model cannot get from context. Most teams in 2026 should start with retrieval and memory, and fine-tune only for behavior that context cannot supply.

The builder decision table

Option Cost Freshness Forgetting risk When to use it
Retrieval (RAG) Low to set up, small cost per query Minutes: update the index None for model skills Facts that change: docs, policies, product data, tickets
Long context Grows with every call Instant: paste the current version None One large document or codebase per task
Memory files and agent memory Low Instant, but only as good as what gets written None for the model, but a careless write can overwrite a good note Preferences, decisions, corrections, project state
Fine-tuning High: data, training, evaluation, re-runs on every model upgrade Slow: days to weeks per cycle High without replay Style, format, tone, narrow skills the base model lacks
Adapters such as LoRA Medium Slow, but you can keep one per task or customer Lower than full fine-tuning, not zero Several distinct behaviors on one base model
Periodic retraining with replay Highest Months Controlled by mixing old data back in Model owners with large new datasets

The fine-tuning versus RAG guide goes deeper on the two most common choices, and the retrieval-augmented generation wiki page defines the terms.

A decision flowchart

yes no yes no no yes no yes Your model needssomething new Is it a fact thatchanges over time? Retrievalor a memory file Does it fit in oneprompt with room to spare? Long contextor instructions Is it a behavior,style or format? Did prompting andexamples already fail? Adapter or fine-tunewith replay and an eval suite
yes no yes no no yes no yes Your model needssomething new Is it a fact thatchanges over time? Retrievalor a memory file Does it fit in oneprompt with room to spare? Long contextor instructions Is it a behavior,style or format? Did prompting andexamples already fail? Adapter or fine-tunewith replay and an eval suite

Retrieval versus fine-tuning, step by step

The two paths differ most in when the new knowledge takes effect and what it can break. The sequence below shows the same correction, "our refund window is now 30 days," delivered both ways.

update the refund policy document what is our refund window? retrieve the current policy 30 days 30 days, with the source add the new policy to a fine-tuning set train, then evaluate for forgetting ship new weights retrieval path, effective at once fine-tuning path, days later old skills must be re-tested Team member Knowledge store Frozen model Training pipeline
update the refund policy document what is our refund window? retrieve the current policy 30 days 30 days, with the source add the new policy to a fine-tuning set train, then evaluate for forgetting ship new weights retrieval path, effective at once fine-tuning path, days later old skills must be re-tested Team member Knowledge store Frozen model Training pipeline

Treat memory like a belief, not a log

Memory outside the weights does not forget model skills, but it can still overwrite itself. Most agent memory systems have one implicit rule: whatever was written last wins. That is the naive single-sample update that Javed says "completely destroys your prior knowledge," moved from the weights to a file. The fix is to borrow the ladder's level 1 and give each stored fact a certainty and a rate of change.

FACT The fact in one sentence string statement frozen, slow, medium or fast string volatility How sure the team is float certainty When a person last confirmed it date last_checked Who can change it string owner EVIDENCE Document, person or system string source A new source or a repeat boolean independent REVISION When it changed date changed_at Why it changed string reason supported by changed by
FACT The fact in one sentence string statement frozen, slow, medium or fast string volatility How sure the team is float certainty When a person last confirmed it date last_checked Who can change it string owner EVIDENCE Document, person or system string source A new source or a repeat boolean independent REVISION When it changed date changed_at Why it changed string reason supported by changed by

Five rules follow from the history in this guide:

  1. Separate facts by volatility. Brand rules and legal terms barely move. This week's priorities move daily. Gate writes to slow facts behind a confirmation, and let fast facts update freely. This is EWC's stiff and loose springs, applied to a document.
  2. Count sources, not repetitions. Ten notes that cite one email are one source. This is level 1's coffee shop and gravity rule.
  3. Keep the old version. Never edit a slow fact in place without a record of what it was. Savings, the faster relearning seen in 1989, only works if something of the old knowledge remains.
  4. Give the reviewer different evidence. A second model that reads the same draft is not a second source. A test result, a rendered page or a real customer reply is.
  5. Schedule a review. Replay works because old items come back before they fade. A weekly review of the oldest unchecked facts is replay for a knowledge base, the same idea that drives AI flashcard makers and AI tutors for people.

Where Taskade Fits: State, Not Learning

Taskade keeps knowledge in the workspace and not in model weights, so new knowledge cannot overwrite an old skill. Your projects, databases, the knowledge you attach to agents and the persistent memory of custom agents hold what your team knows, and AI agents read that knowledge each time they work. By Sutton's definition, that is state, not learning. Taskade's models do not learn from your workspace, and Taskade does not claim they do.

That plain statement is also the practical advantage. A fine-tuned model can lose general skills when it learns your refund policy. A workspace cannot, because the model never changes. When the policy changes, you change the project, and every agent that reads it has the new version on its next run. The Workspace DNA architecture describes the loop: Memory in projects and databases, Intelligence in AI agents that reason over that memory, and Execution in automations that act and write results back.

MEMORYprojects, databases,attached knowledge INTELLIGENCEAI agents read the memoryon every run EXECUTIONautomations act andwrite results back MEMORY, updateda new fact overwritesno model skill A person reviews and correctsthe judgment layer
MEMORYprojects, databases,attached knowledge INTELLIGENCEAI agents read the memoryon every run EXECUTIONautomations act andwrite results back MEMORY, updateda new fact overwritesno model skill A person reviews and correctsthe judgment layer

A Taskade knowledge base that AI agents read from, with projects that hold a team's facts outside any model's weights

The ladder also says where people belong. A workspace supplies memory, which is level 0 in a form that never forgets a model skill. It does not supply machine metacognition. People do. Taskade automations can stop at a Wait for approval step and hold until a teammate says yes before they send money, mail or a post. The automation does the middle part, and a person makes the call the CRUX agents could not make on their own.

Ladder level What a Taskade workspace gives you What stays with your team
0. Replay Knowledge in projects, not weights, so nothing a model knows is overwritten Deciding what to store
1. Selective learning A place to record sources, owners and review dates for each fact Deciding how sure you are
2. Metacognition Approval steps that pause an automation for a person's judgment Asking whether the question is right
3. Intuition Published apps in the Taskade App Factory to start from a working structure Knowing which template fits
4. Instinct Goals written into projects and agent instructions Deciding what matters
5. Community Shared workspaces where teammates review each other's work The review itself

Connecting a file to a Taskade AI agent so it can answer from the team's knowledge

To try the pattern in this guide, connect a source to an agent with agent knowledge, or describe the system you want and let Taskade Genesis build the projects, agents and automations together. On Free, you earn free AI credits with referrals and invites, and you can upgrade any time.


Glossary

Term Definition
Adapter A small set of added parameters trained on a frozen base model, such as LoRA
Catastrophic forgetting The abrupt loss of old skills when a network trains on new data without the old
Catastrophic interference The original 1989 name for catastrophic forgetting
Class-incremental learning A scenario in which the model must tell apart classes from all past tasks with no task label
Complementary learning systems The 1995 theory of a fast hippocampus and a slow neocortex linked by replay
Consolidation The process that turns a fragile new memory into a stable long-term one
Continual backpropagation Backpropagation that keeps reinitializing a small fraction of the least useful units
Continual learning Learning from a stream of data without erasing earlier knowledge. Also called lifelong or incremental learning
Elastic weight consolidation (EWC) A penalty that pulls important weights back toward their old values
Experience replay Storing past experience and reusing it during training
Fisher information A measure of how much a model's predictions depend on a given weight
Generative replay Replaying samples from a trained generator instead of stored data
Loss of plasticity The gradual decline in a network's ability to learn new tasks
Metacognition Monitoring and controlling your own thinking, including whether the question is right
Pseudo-rehearsal Replaying the old network's answers to random inputs instead of the original data
Savings Faster relearning of something that was learned and then forgotten
Stability-plasticity dilemma The trade-off between keeping old knowledge and learning new knowledge
State Sutton's term for what context changes: the model's input, not the model
Test-time training Updating some parameters during inference, on the input at hand

The persistent memory wiki page covers how agent memory persists between sessions in more detail.


Frequently Asked Questions

What is catastrophic forgetting in simple terms?

Catastrophic forgetting is what happens when a neural network learns something new and loses something old. The network stores every skill in one shared set of weights, so the weight changes that teach the new skill also damage the old one. Michael McCloskey and Neal Cohen named the problem in 1989 after a small network learned the twos addition facts and lost the ones facts it had already mastered.

Is catastrophic forgetting the same as catastrophic interference?

Yes. The two terms name the same effect. McCloskey and Cohen used catastrophic interference in their 1989 paper, and later work, including Robert French's 1999 review, popularized catastrophic forgetting. Interference names the cause, new learning that disturbs old weights. Forgetting names the result, the lost performance on the old task.

Who discovered catastrophic forgetting?

Michael McCloskey and Neal J. Cohen reported it in 1989 in Psychology of Learning and Motivation, volume 24. Roger Ratcliff reported the same effect in recognition-memory models in Psychological Review in 1990. Stephen Grossberg had already named the underlying trade-off, the stability-plasticity dilemma, in 1980.

Why can't AI learn continuously?

Three reasons stack up. New weight updates overwrite old knowledge, which is catastrophic forgetting. Networks trained on a long stream of tasks lose the ability to learn at all, which is loss of plasticity. And a model that changes after release is hard to test, hard to keep safe and expensive to run for each user. So labs freeze the weights at release and add memory outside the model instead.

How does the brain avoid catastrophic forgetting?

The leading account is complementary learning systems theory from 1995. The hippocampus learns new episodes fast with sparse, separate codes. The neocortex learns slowly. Replay, including replay during sleep, feeds new memories to the neocortex interleaved with old ones, so the slow system can absorb new knowledge without overwriting old structure.

What is elastic weight consolidation?

Elastic weight consolidation, or EWC, is a 2017 DeepMind method that protects the weights an old task needs. After a task, it estimates how important each weight was with the Fisher information. During the next task, a penalty pulls important weights back toward their old values, like stiff springs, while unimportant weights move freely. It works well when the task is known at test time and poorly when the model must work out which task it faces.

What is loss of plasticity?

Loss of plasticity is the gradual decline in a network's ability to learn anything new after long training on a stream of tasks. Dohare and colleagues reported it in Nature in August 2024. In their preprint's Continual ImageNet test, accuracy fell from about 89 percent on an early task to about 77 percent by the 2,000th task, close to a linear model. Their fix, continual backpropagation, keeps reinitializing a small fraction of the least useful units.

Do large language models suffer from catastrophic forgetting?

Yes, when you change their weights. A 2023 study observed forgetting in general across models from 1 billion to 7 billion parameters, during continual instruction tuning. Fine-tuning on a narrow dataset can erode general skills. This is one reason frontier models ship with frozen weights and get new knowledge through retrieval, long context and memory files instead.

Does LoRA prevent catastrophic forgetting?

LoRA reduces forgetting but does not prevent it. A 2024 study titled "LoRA Learns Less and Forgets Less" found that low-rank adapters kept more of the base model's general skills than full fine-tuning, and also learned less of the new domain. A 2025 Meta study measured forgetting on a held-out benchmark of 89 percent after full fine-tuning, 71 percent with LoRA and 11 percent with sparse memory fine-tuning.

Is RAG or agent memory a form of continual learning?

Not in the strict sense. Retrieval, long context and memory files change what the model reads, not the model itself. Richard Sutton puts it this way: you can change the state of the model with more context, but the model is not learning. These tools are still the most practical way to give an AI system new knowledge today, because they overwrite no old skill.

Does Taskade's AI learn from my workspace?

Taskade keeps your knowledge in the workspace, not in model weights. Projects, databases, attached knowledge and the persistent memory of custom agents hold what your team knows, and agents read that knowledge each time they work. Because no weight changes, a new fact cannot overwrite an old skill. By Sutton's definition that is state, not learning, and Taskade describes it that way.

Will continual learning be solved soon?

Researchers disagree. Some researchers, including one at Anthropic, predicted that continual learning would be solved in 2026, and new methods such as Titans, Nested Learning and sparse memory fine-tuning reduce forgetting in tests. Critics say these methods make models forget a little less rather than learn continually. The learning-ladder view adds that replay and forgetting are only the first rung, and that judging what is worth learning is harder still.



Before you go: A model that learns your business today can lose a skill it had yesterday. A workspace keeps your knowledge where no update can erase it. Taskade Genesis turns one prompt into projects that hold your memory, AI agents that reason over it and automations that act and write results back. Explore what others built in the Taskade App Factory, or start from the Taskade home page.


Sources

  1. McCloskey, M., and Cohen, N. J. (1989). Catastrophic Interference in Connectionist Networks: The Sequential Learning Problem. Psychology of Learning and Motivation, 24, 109 to 165.
  2. Ratcliff, R. (1990). Connectionist models of recognition memory: Constraints imposed by learning and forgetting functions. Psychological Review, 97(2), 285 to 308.
  3. French, R. M. (1999). Catastrophic forgetting in connectionist networks. Trends in Cognitive Sciences, 3(4), 128 to 135.
  4. McClelland, J. L., McNaughton, B. L., and O'Reilly, R. C. (1995). Why there are complementary learning systems in the hippocampus and neocortex. Psychological Review, 102(3), 419 to 457.
  5. Robins, A. (1995). Catastrophic Forgetting, Rehearsal and Pseudorehearsal. Connection Science, 7(2), 123 to 146.
  6. Wilson, M. A., and McNaughton, B. L. (1994). Reactivation of Hippocampal Ensemble Memories During Sleep. Science, 265, 676 to 679.
  7. Lin, L.-J. (1992). Self-improving reactive agents based on reinforcement learning, planning and teaching. Machine Learning, 8, 293 to 321.
  8. Kirkpatrick, J., et al. (2017). Overcoming catastrophic forgetting in neural networks. PNAS, 114(13), 3521 to 3526.
  9. Huszár, F. (2018). Note on the quadratic penalties in elastic weight consolidation. PNAS, 115(11).
  10. Zenke, F., Poole, B., and Ganguli, S. (2017). Continual Learning Through Synaptic Intelligence. ICML 2017.
  11. Rusu, A. A., et al. (2016). Progressive Neural Networks. arXiv:1606.04671.
  12. Shin, H., Lee, J. K., Kim, J., and Kim, J. (2017). Continual Learning with Deep Generative Replay. NeurIPS 2017.
  13. van de Ven, G. M., and Tolias, A. S. (2019). Three scenarios for continual learning. arXiv:1904.07734.
  14. Dohare, S., et al. (2024). Loss of plasticity in deep continual learning. Nature, 632, 768 to 774. Preprint: arXiv:2306.13812.
  15. Aitchison, L., et al. (2021). Synaptic plasticity as Bayesian inference. Nature Neuroscience, 24, 565 to 571.
  16. Luo, Y., et al. (2023). An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuning. arXiv:2308.08747.
  17. Biderman, D., et al. (2024). LoRA Learns Less and Forgets Less. TMLR.
  18. Shenfeld, I., Pari, J., and Agrawal, P. (2025). RL's Razor: Why Online Reinforcement Learning Forgets Less. ICLR 2026.
  19. Lin, J., et al. (2025). Continual Learning via Sparse Memory Finetuning. arXiv:2510.15103.
  20. Sun, Y., et al. (2020). Test-Time Training with Self-Supervision for Generalization under Distribution Shifts. ICML 2020.
  21. Behrouz, A., Zhong, P., and Mirrokni, V. (2024). Titans: Learning to Memorize at Test Time. arXiv:2501.00663.
  22. Behrouz, A., et al. (2025). Nested Learning: The Illusion of Deep Learning Architectures. NeurIPS 2025.
  23. Dupoux, E., LeCun, Y., and Malik, J. (2026). Why AI systems don't learn and what to do about it. arXiv:2603.15381.
  24. Kirgis, P., Kapoor, S., et al. (2026). Can AI agents conduct open-ended AI research? Early evidence from two case studies. arXiv:2607.27191.
  25. Amit, D. J., Gutfreund, H., and Sompolinsky, H. (1985). Storing infinite numbers of patterns in a spin-glass model of neural networks. Physical Review Letters, 55, 1530.
  26. Chase, W. G., and Simon, H. A. (1973). Perception in chess. Cognitive Psychology, 4(1), 55 to 81.
  27. JT Yu, Universal Resilience (November 2, 2026). AI Has Finally Hit the Brain's Hardest Problem. YouTube.
  28. Rich Sutton and Khurram Javed. Why AI Models Stop Learning, and How to Start It Again. YouTube.
  29. Andrej Karpathy on the Dwarkesh Podcast (October 2025). Interview.
  30. Celia Ford, Transformer (January 22, 2026). Teaching AI to learn.
  31. Sean Goedecke (February 23, 2026). What's so hard about continuous learning?
  32. Wikipedia. Catastrophic interference.

▲ ■ ● Memory. Intelligence. Execution. Keep what works, change what does not.

Build this in Taskade

Open this prompt in Taskade Genesis, edit it, and run it when it reads right:

Build a team knowledge base that never overwrites what works: a facts table with a source, a certainty score and a last-checked date for each fact, an AI agent that compares each new fact with the existing entries and flags conflicts before it saves anything, and a weekly automation that sends the oldest unchecked facts to their owner for review.

Start with this prompt

View this page as Markdown

0%

On this page

Catastrophic Forgetting at a GlanceWhat Is Catastrophic Forgetting?Why Does Catastrophic Forgetting Happen?The History of Catastrophic Forgetting, 1989 to 2026The master timeline1989: the network that forgot the ones facts1990: Ratcliff and recognition memoryThe 1990s fixes1999: French's reviewWhat Does the Brain Do Instead? Complementary Learning SystemsThe evidence for replayThe human version of replayHuman versus machine, side by sideReplay Comes to Machines: Experience Replay and Generative Replay1992: Lin's experience replay2015: the DQN replay buffer2017: generative replay and exemplar memoryElastic Weight Consolidation and the Regularization FamilyWhere the sources disagreeThe rest of the familyGrowing the Network: Progressive Networks and Parameter IsolationThe Method Families ComparedThe Three Continual-Learning ScenariosLoss of Plasticity: The Second ProblemWhat helped and what hurtThe fix: continual backpropagationCatastrophic Forgetting in Large Language ModelsAdapters reduce forgetting and learning togetherHow you train matters as much as what you trainSparse updates forget far lessThe 2025 to 2026 State: Memory Outside the WeightsTest-time training and memory that updates during inferenceContext is state, not learningWhy Can't AI Learn Continuously Yet?The Learning Ladder: Six Levels of Continual LearningLevel 0: replay, the rung AI has climbedLevel 1: selective learning, or how much to updateLevel 2: metacognition, or is the question right?Level 3: intuition, or what is worth a lookLevel 4: instinct, or what matters at allLevel 5: community, or many mindsWhy each level learns from the level below, and why that makes it hardThe 2026 Field Evidence: The CRUX StudyThe Navier-Stokes claim, with its caveatsWhat Builders Can Do TodayThe builder decision tableA decision flowchartRetrieval versus fine-tuning, step by stepTreat memory like a belief, not a logWhere Taskade Fits: State, Not LearningGlossaryFrequently Asked QuestionsSources

Related Articles

An archivist on a ladder files a labeled box in the stacks of the Washington National Records Center. Each box has a type, a label and a place, like a concept file in an Open Knowledge Format bundle. Public domain photo, U.S. National Archives.
November 16, 2026AI

Open Knowledge Format (OKF) Explained (2026): Every Rule in Google's Spec for Agent-Readable Knowledge

OKF is a folder of Markdown files with one required field. This guide covers every rule in Google's v0.2 spec, what chan...

Vannevar Bush at his desk in the early 1940s. His 1945 memex essay described a personal knowledge store with linked trails, the idea Andrej Karpathy cites as the ancestor of the LLM wiki. Public domain photo, Library of Congress.
November 13, 2026AI

LLM Wiki Explained (2026): Karpathy's Pattern, Google's OKF, and When a Folder Beats RAG

An LLM wiki is a folder of linked Markdown files that an AI agent writes and maintains. How Karpathy's pattern and Googl...

Agentic AI glossary 2026: a fluorescence microscopy image of a dense network of cortical neurons, the biological network that gave AI words like neuron, attention, and memory
September 25, 2026AI

The Agentic AI Glossary (2026): 200+ AI Agent, LLM, and Brain Terms Explained

The agentic AI glossary for 2026: 200+ AI agent, LLM, security, and brain-science terms in 13 families, each with a plai...

The Guidonian hand, a medieval memory aid for music theory, from a late 15th-century manuscript (Wikimedia Commons, public domain), shown above a guide to AI music theory tools
November 20, 2026AI

Best AI Music Theory Tools 2026: 20 Tutors and Ear Trainers

20 AI music theory tools compared: chat tutors, ear trainers, chord tools and practice apps for every level. Free plans ...

Human computers at work in the NACA High-Speed Flight Station computer room in 1949, when computer was a job title. Photo by NACA and NASA, public domain, via Wikimedia Commons
November 19, 2026AI

What Is an AI Employee? Roles, Costs, Risks and How to Hire Your First One (2026)

An AI employee is an AI agent with a role, memory, a schedule and a manager. See what one does, what it costs, what goes...

How to build an AI support agent for your website: a row of Chesapeake and Potomac Telephone Company operators at a switchboard in 1927, an early human handoff desk. Photo: U.S. National Archives (NARA 522875) via Wikimedia Commons, public domain
November 19, 2026AI

How to Build an AI Support Agent for Your Website (2026)

Put an AI support agent on your website that answers from your docs, hands hard questions to a person, and logs each han...

View All Articles