Open-Source LLM History: GPT-2 to Kimi K3 (2026)

Open-source large language models went from "too dangerous to release" to larger and more capable than most people thought possible in the open — in about six years. In 2019, OpenAI held back the full weights of GPT-2, a 1.5-billion-parameter model, worried it was too risky to publish. In July 2026, a Chinese lab shipped Kimi K3, a 2.8-trillion-parameter open-weight model at frontier level, with the full weights posted on Hugging Face on July 27 for anyone to download. Three months later, even Meta, which had gone closed in April 2026, came back with the 816-billion-parameter Muse Spark Open.

How did we get from one to the other? Which models actually mattered, who built them, and what's the real difference between "open source" and "open weight" — a distinction almost every article gets wrong? This is the complete timeline. 🧬

TL;DR: Open-source LLMs evolved from GPT-2's cautious 2019 release → the truly-open era of EleutherAI and BLOOM (2021–22) → Meta's LLaMA (2023) detonating the ecosystem → Mistral bringing mixture-of-experts to open weights → DeepSeek's 2025 cost shock → a Chinese open-weight surge (Qwen, Kimi, GLM) that shrank the gap to the closed frontier to a matter of months. Most "open" models are technically open weight, not fully open source. See the 10 best open-source LLMs ranked →

What Is the History of Open-Source LLMs?

The history of open-source LLMs runs from OpenAI's staged release of GPT-2 in 2019 (1.5 billion parameters) through EleutherAI and BLOOM (2021 to 2022), Meta's LLaMA (2023), Mistral's open mixture-of-experts models, DeepSeek-R1 (January 2025) and OpenAI's gpt-oss (August 2025), to the 2026 trillion-parameter era of Kimi K3, Qwen3.8 and Muse Spark Open.

In numbers: the largest open model grew from 1.5 billion parameters in November 2019 to 2.8 trillion in July 2026, an increase of more than 1,800 times in under seven years. The licenses moved the other way, from "too risky to publish" to MIT and Apache 2.0, and then in 2026 to MIT-style licenses with one extra clause for very large API businesses. This guide covers each era in order, with the exact dates, parameter counts and licenses taken from each lab's own announcement, model card or paper.

Open-Source LLM Timeline at a Glance: Every Major Release, 2019 to 2026

This table is the short version of the whole story. Each row is a release that changed what "open" meant. "Total / active" shows the full parameter count and, for mixture-of-experts models, the part that runs for each token.

Sources: each lab's announcement, paper or Hugging Face model card. DeepSeek moved V3 to MIT with the V3-0324 update in March 2025. For the companies behind these rows, read the DeepSeek history, the Qwen history and the Moonshot AI and Kimi history.

What "Open-Source LLM" Actually Means (and Why "Open Weight" Is More Accurate)

Most models people call "open source" are actually open weight — a crucial difference. An open-weight model releases its trained parameters so you can download, run, and fine-tune it. A truly open-source model, by the Open Source Initiative's 2024 definition, releases the weights plus the training code, information about the data, and a license with no usage restrictions.

The Linux Foundation's Model Openness Framework — created by its Global CTO of AI, Matt White — grades this on three levels, and it's the cleanest way to think about the spectrum:

Almost every "open" model you've heard of — Llama, DeepSeek, Qwen, Kimi — sits in that bottom row: open weight. That's still enormously valuable (you can run and adapt them), but it's not the same as fully reproducible. Matt White's own worry, after touring China's labs in 2026, was about disclosure shrinking:

"The research papers have shrunk and then become technical reports. And so there's not enough disclosure in a lot of these technical reports to be able to go and replicate the research."

— Matt White, Linux Foundation

What Is the OSI Open Source AI Definition (OSAID)?

The Open Source AI Definition 1.0 is the Open Source Initiative's rule for when an AI system counts as open source. The OSI published it on October 28, 2024, after two years of drafts. It applies the four classic software freedoms to AI: anyone must be able to use the system for any purpose, study how it works, modify it, and share it, with or without changes.

To make those freedoms real, the definition asks for the "preferred form to make modifications." For a machine-learning model that means three parts, all under OSI-approved terms:

• Parameters: the weights, and ideally checkpoints from key training stages.
• Code: the complete code used to process data, train, and run the model.
• Data information: a description of the training data detailed enough that a skilled person could build a substantially equivalent system, with a list of public and third-party sources.

The data rule is the one most labs fail. The definition does not demand that every training document be published, because some data cannot be shared legally. It does demand enough detail to rebuild something equivalent. The full text on opensource.org is short and worth reading.

Which Famous Models Pass the OSI Definition?

The OSI does not certify models one by one. During its validation phase it did test a set of real models against the draft, and the results show how rare full openness is:

Source: the OSI's OSAID FAQ. In February 2025 the OSI also wrote that the Llama licenses are not open source because they restrict who can use the model and for what. That is why this guide says "open weight" for most of the famous models and saves "open source" for the few that publish everything. For a plain-language glossary, see our wiki entries on open-source models, model parameters and large language models.

What Does "Open Weights" Look Like on Disk?

Andrej Karpathy's well-known talk describes an open-weight model in the simplest terms: a parameters file and a small program that runs it. In his example, Llama 2 70B is a file of about 140 GB, since each of its 70 billion parameters takes 2 bytes, plus a few hundred lines of code. Everything that makes the model useful lives in that one file, which is why releasing it, or not, is such a big decision. The first part of the talk explains it in a few minutes.

Don't just read the open-source timeline — clone a living dashboard that tracks every model, license, and benchmark in minutes. One click, no code, free.

A Visual Timeline of Open-Source LLMs (2019 → 2026)

Here's the whole arc at a glance. Each era changed what "open" could mean — from a cautious weights-withheld release to trillion-parameter models anyone can download.

Last updated: October 2026 — refreshed for Kimi K3 (July 16, 2026) and its July 27 open-weight release, plus DeepSeek V4 and the latest Qwen and GLM releases. Model dates, parameter counts, and licenses move fast and vary by source; where figures are disputed we say so. Check Wikipedia and each vendor's model card before quoting exact specs.

Before GPT-2: When Open Research Was the Default (2013 to 2018)

Before 2019, nobody asked whether a language model was "safe to release." Research labs published papers, code and weights as a matter of course, and the field moved fast because of it.

• 2013: Google released word2vec as open-source code with pretrained word vectors. It was the first time many developers used a model that "knew" what words meant.
• June 2017: eight Google researchers published Attention Is All You Need, the paper that introduced the transformer, the architecture every model in this history uses. Read our wiki entry on the transformer and the attention mechanism for the basics.
• June 2018: OpenAI published GPT-1, a 117-million-parameter transformer, together with its code and weights.
• November 2018: Google open-sourced BERT with weights under Apache 2.0. BERT reads text instead of writing it, so it is not a chatbot ancestor, but it proved that one pretrained model could be fine-tuned for hundreds of tasks.

So "open" was the starting point. The surprise of 2019 was a lab deciding to close a door that had always been open. For the wider arc, read our history of computing and the story of GPT from GPT-1 to GPT-6.

2019: GPT-2 and the Cautious Birth of Open Models

The modern open-model story starts with a decision not to release. On February 14, 2019, OpenAI announced GPT-2, a 1.5-billion-parameter model described in the paper Language Models are Unsupervised Multitask Learners, but held back the full weights, worrying openly that the model could be used to mass-produce fake text. It released the full 1.5B weights only on November 5, 2019, after a staged rollout and a partner study, published in August 2019 as Release Strategies and the Social Impacts of Language Models, found no strong evidence of misuse.

The myth-correction matters: GPT-2 was not "fully open" at launch. It was a careful, staged release — the first time a major lab treated a language model as something whose openness was a policy question, not a default. That framing has shaped every open-vs-closed debate since. (If you want the mechanics of how these models work at all, our explainer on how large language models work is the primer.)

When Was GPT-2 Released? The Four-Stage Rollout

GPT-2 was released in four stages over nine months in 2019, from the 124-million-parameter model on February 14 to the full 1.5-billion-parameter model on November 5. OpenAI published each size with a blog post, and it shared the larger sizes with research partners before the public got them.

The "first reported as" column explains a common source of confusion. The original paper listed 117M, 345M, 762M and 1,542M. OpenAI later corrected the counts to 124M, 355M, 774M and 1,558M, which is why you will see both sets of numbers quoted for the same models.

How Many Parameters Does GPT-2 Have?

GPT-2 has 1.5 billion parameters in its full size, 1,558 million to be exact. It is a decoder-only transformer with 48 layers and a 1,024-token context window. OpenAI trained it on WebText, about 40 GB of text from roughly 8 million web pages that Reddit users had linked to.

Why Did OpenAI Hold GPT-2 Back?

OpenAI held GPT-2 back because it worried the model could produce convincing fake news, spam and abuse at a scale no one had seen. The February 2019 post said the lab would not release "the dataset, training code, or GPT-2 model weights" for the largest model, and called the plan "an experiment in responsible disclosure."

The partner research that followed gave the decision some data. Researchers at Cornell asked people to rate the credibility of GPT-2 text: the 1.5B model scored 6.91 out of 10, the 774M model 6.72, and the 355M model 6.07. OpenAI also built a detector that caught about 95% of 1.5B outputs. In the end, the lab said it had seen "no strong evidence of misuse" and published everything.

Two things are worth noticing. First, the debate GPT-2 started, about whether a model is too dangerous to share, is the same debate every lab still has about frontier weights. Second, the license OpenAI chose was a Modified MIT License that asked users to say when content came from GPT-2. Six years later, Moonshot AI shipped Kimi K2 under a modified MIT license with an attribution clause of its own. The idea came full circle.

GPT-2's Afterlife: From $43,000 to $672

GPT-2 is still the standard teaching model for open LLMs. Andrej Karpathy has estimated that OpenAI's original run used 32 TPU v3 chips for about a week, roughly $43,000 of compute. In July 2024 his open-source llm.c project reproduced the full 1.5B GPT-2 on one 8x H100 node in 24 hours for about $672, thanks to better GPUs, better software and cleaner data. In August 2019, two Brown University graduate students had already shown the "too dangerous" model could be rebuilt from public information, when they released their own 1.5B replication, OpenGPT-2.

That cost curve is the hidden engine of this whole history: every year, the model that once needed a frontier lab gets cheaper for everyone. Karpathy's four-hour walkthrough rebuilds the 124M GPT-2 from an empty file.

2021–2022: The Truly-Open Era — EleutherAI, OPT, and BLOOM

The first genuinely open GPT-3-class models didn't come from a big lab — they came from a volunteer collective. EleutherAI released GPT-Neo (March 2021) and GPT-J-6B, then GPT-NeoX-20B (2022), reverse-engineering GPT-3-style capability and publishing everything. For the first time, anyone could download a large, capable model and its code.

Two 2022 releases pushed openness to its high-water mark:

• Meta OPT-175B — Meta's attempt to open-source a GPT-3-scale model, released with training logs and a research license.
• BLOOM — the crown jewel of the truly-open era. Built by BigScience, a collaboration of over 1,000 researchers, BLOOM was a 176-billion-parameter model trained on 46 natural languages and 13 programming languages, released with open weights, code, and full documentation. By the Model Openness Framework, BLOOM is one of the few models that reaches "Open Science" — the genuinely fully-open tier.

EleutherAI and The Pile: A Discord Server Rebuilds GPT-3

EleutherAI started in July 2020 as a Discord server, founded by Connor Leahy, Sid Black and Leo Gao, with one goal: build an open replication of GPT-3, which OpenAI had announced that summer as an API only. Its first big contribution was not a model but a dataset. The Pile, released on December 31, 2020, packed 825 GiB of text from 22 sources, including books, papers, code and web text, and it became the standard open training set for the next three years.

The models came next, each one larger:

• GPT-Neo (March 2021): 1.3B and 2.7B parameters, trained on The Pile, MIT license.
• GPT-J-6B (June 2021): built by Ben Wang and Aran Komatsuzaki on Google TPUs with the Mesh Transformer JAX library, Apache 2.0. For about a year it was the most capable model anyone could download.
• GPT-NeoX-20B (February 2022): 20 billion parameters, Apache 2.0, trained on GPUs donated by the cloud company CoreWeave.
• Pythia (2023): a suite of 16 models from 70 million to 12 billion parameters, each released with intermediate checkpoints, built so scientists could study how a model learns over training. Pythia is one of the models that passed the OSI definition.

OPT-175B: Big Tech Opens a GPT-3-Sized Model

Meta released OPT (Open Pretrained Transformer) on May 3, 2022. Sizes from 125 million to 66 billion parameters were free to download, and the full 175B model was available to researchers on request under a noncommercial license. Meta also published a logbook of the training run, with every crash and restart, which was rare honesty for a large lab. OPT matched GPT-3 on many benchmarks, but its license kept it inside research.

BLOOM: 1,000 Researchers and a French Supercomputer

BLOOM, released on July 12, 2022, is the high-water mark of the truly open era. The BigScience workshop, coordinated by Hugging Face, brought together more than 1,000 researchers from dozens of countries. They trained the 176-billion-parameter model for about four months on 384 Nvidia A100 GPUs in the Jean Zay supercomputer near Paris, with compute time granted by the French government.

BLOOM covered 46 natural languages and 13 programming languages, including languages such as Swahili, Yoruba and Vietnamese that most large models of the time left out. It shipped under the BigScience RAIL License 1.0, an early "responsible AI" license that allowed commercial use but banned a list of harmful uses. That use-based list is why the OSI later said BLOOM would pass its definition only with license changes.

GLM-130B and Galactica: Two Endings for 2022

Two more releases closed the year. In August 2022, Tsinghua University released GLM-130B, a 130-billion-parameter bilingual Chinese and English model. Its team later founded Zhipu AI, now Z.ai, which makes the GLM models that sit near the top of the open rankings in 2026. In November 2022, Meta released Galactica, a 120B model for science, with a public demo. Users quickly showed it inventing confident, fake citations, and Meta took the demo down after three days. The lesson about AI hallucinations stuck.

This era proved the community could build at scale. What it hadn't yet proved was that open models could be genuinely useful. That changed in 2023.

2023: Llama Changes Everything (and the "Open" Debate Begins)

Meta's LLaMA is the single most important release in this history, because it turned open weights into an ecosystem. LLaMA 1 (February 24, 2023) came in 7B to 65B sizes and — after the weights leaked online — detonated a wave of community fine-tunes (Alpaca, Vicuna, and hundreds more). Llama 2 (July 2023) made it official, shipping with a license that permitted commercial use.

Why ChatGPT Made Open Models Urgent

ChatGPT launched on November 30, 2022, and changed the question. Before it, open models were research tools. After it, every company wanted a chat model of its own, and most did not want to send their data to one vendor's API. That demand is what turned the next open release into an event. For the closed side of the same story, read the history of OpenAI and ChatGPT.

Did LLaMA Leak? The 4chan Torrent of March 2023

Yes. Meta announced LLaMA (Large Language Model Meta AI) on February 24, 2023, in four sizes: 7B, 13B, 33B and 65B parameters. The two larger models trained on 1.4 trillion tokens of public data, and Meta's paper reported that the 13B model beat the 175B GPT-3 on most benchmarks. Access was by application only, under a noncommercial research license.

On March 3, 2023, about a week later, someone posted a torrent of the weights on 4chan. Within days the files were everywhere, and Meta's takedown requests could not pull them back. The leak did what no press release could. It put a strong base model in the hands of every hobbyist, student and startup at the same moment.

The Fine-Tune Explosion: Alpaca, Vicuna, llama.cpp and Dolly

What followed was the fastest burst of open AI work up to that point. Most of it took days, not months, and cost less than a used laptop.

In May 2023 a Google engineer's internal memo leaked under the title "We Have No Moat, And Neither Does OpenAI." Its argument was that open-source fine-tunes were closing the gap faster than any big lab could. The memo overstated the case, but it captured the mood. For the first time, the open community was setting the pace.

Falcon, MPT and the First Truly Commercial Open Models

LLaMA's license did not allow commercial use, so the first open models a business could ship came from elsewhere. MosaicML's MPT-7B arrived in May 2023 under Apache 2.0. The Technology Innovation Institute in Abu Dhabi released Falcon-40B the same month. It went to the top of the Hugging Face Open LLM Leaderboard, and on May 31, 2023 TII moved it to Apache 2.0. Its larger sibling, Falcon 180B, followed in September 2023 under a custom TII license.

Hugging Face became the meeting place for all of it. Its model hub, its leaderboard and its open-source Transformers library turned "download a model" into one line of code. Our guide to fine-tuning versus RAG covers what all those community fine-tunes were trying to do.

Llama 2: Open Weights Go Commercial (July 18, 2023)

Llama 2 arrived on July 18, 2023, with Microsoft as a launch partner. It came in 7B, 13B and 70B sizes, trained on 2 trillion tokens with a 4,096-token context window, plus chat versions tuned with human feedback. The big change was the license. The Llama 2 Community License allowed commercial use, with one headline exception: a company whose products had more than 700 million monthly active users in the month before release had to ask Meta for a separate license. In practice, that clause excluded only a handful of rivals.

Code Llama followed on August 24, 2023. By the end of the year, "built on Llama" was the most common line in an open-model card.

But Llama also started the argument that still runs today.

Is Llama Really Open Source?

No, not by the strict definition. The Open Source Initiative classifies Llama as source-available or open weight, not open source, for two reasons: the Llama Community License restricts use above 700 million monthly active users, and it ships with an acceptable-use policy plus undisclosed training data. Critics call this "openwashing" — marketing a restricted release as open. Defenders point out that for 99.9% of developers, Llama was effectively open and did more to democratize LLMs than any purely-open model. Both are true, which is why the debate never fully resolves.

2023–2024: Mistral and the Mixture-of-Experts Moment

A French startup made the next leap. Mistral AI, founded by ex-Meta and DeepMind researchers Arthur Mensch, Guillaume Lample, and Timothée Lacroix, released Mistral 7B (September 2023) under the fully-permissive Apache 2.0 license — no MAU clause, no strings. Then came Mixtral 8x7B (December 2023), which brought mixture-of-experts (MoE) to open weights: instead of running the whole model for every token, MoE activates only a few specialist "experts," delivering big-model quality at small-model cost.

MoE is the architecture that made everything after it possible. Every trillion-parameter open model since — DeepSeek, Kimi, GLM — is an MoE. Mistral proved you could ship it in the open, and that Europe had a frontier lab of its own.

The Magnet Link Release

Mistral AI was founded in Paris in April 2023. Arthur Mensch came from Google DeepMind, and Guillaume Lample and Timothée Lacroix came from Meta, where they had worked on LLaMA. Five months later, on September 27, 2023, the company released Mistral 7B in an unusual way: a post on X with nothing but a BitTorrent magnet link. No blog, no demo, no waitlist.

The 7.3-billion-parameter model beat Llama 2 13B on Mistral's reported benchmarks, and it shipped under Apache 2.0, with no user threshold and no use policy. On December 8, 2023, Mistral posted another bare magnet link. This one was Mixtral 8x7B: eight expert networks per layer, two chosen for each token, 46.7 billion parameters in total and about 12.9 billion used per token. Mistral reported that it matched or beat Llama 2 70B and GPT-3.5 on most benchmarks while running at the speed of a much smaller model.

Mistral's License Path After Mixtral

Mistral's later releases show how unsettled open licensing still was. Mixtral 8x22B (April 2024, 141B total, 39B active) stayed on Apache 2.0. Mistral Large 2 (July 2024) moved to a research-only license for its weights. Then the company swung back: Mistral Small 3 (January 2025) and Mistral Large 3 (December 2025, 675 billion total and 41 billion active parameters) both shipped under Apache 2.0, and Mistral Small 4 (2026, 119B total, 6.5B active) did too. Mistral Medium 3.5 (2026), a dense 128B model, uses a modified MIT license with an exception for companies with large revenue. For a head-to-head view, see Mistral vs Llama.

How Mixture of Experts Took Over Open Models

A mixture-of-experts (MoE) model splits each layer's feed-forward network into many smaller "experts" and adds a small router that sends each token to only a few of them. The model can store a huge amount of knowledge in its total parameters while each token pays the compute cost of only the active ones. That trade is why almost every open flagship released since 2025 is an MoE. The last famous dense giant was Llama 3.1 405B in July 2024. Read our wiki entry on mixture of experts for the full mechanics.

The open MoE lineage got sparser every year. Mixtral used a quarter of its parameters per token. Kimi K3 uses less than 4%. That shrinking fraction is the single biggest reason a 2.8-trillion-parameter model can be served at a price close to a mid-size closed model.

2024: The Field Scales Up and Matures

By 2024 the open ecosystem professionalized. Meta shipped Llama 3, 3.1 (a 405-billion-parameter flagship), 3.2 (multimodal), and 3.3. Google entered with Gemma. And Alibaba's Qwen family began its rise to become the most prolific open-weight lineage of all — dozens of models across sizes and specialties under Apache 2.0.

Llama 3 and 3.1: The First Open Model in the GPT-4 Class

Llama 3 (April 18, 2024) came in 8B and 70B sizes, trained on more than 15 trillion tokens, seven times the data of Llama 2. Llama 3.1 (July 23, 2024) added the 405-billion-parameter flagship, a 128,000-token context window, and a license that let developers use its outputs to train other models. Meta trained the 405B model on more than 16,000 H100 GPUs and presented it as the first openly available model in the class of GPT-4o and Claude 3.5 Sonnet.

The same day, Mark Zuckerberg published a letter titled "Open Source AI Is the Path Forward". He argued that open models would follow the path of Linux: cheaper, safer through public scrutiny, and impossible for one company to control. It was the high point of Meta's open strategy. Llama 3.2 (September 25, 2024) added small 1B and 3B models for phones and 11B and 90B vision models, and Llama 3.3 70B (December 6, 2024) closed the year.

The 2024 Class: Gemma, OLMo, Grok-1, DBRX and DeepSeek-V2

2024 was also the year open weights stopped being a Meta story. Almost every major lab shipped at least one:

DeepSeek-V2 is the quiet hinge of this list. Its multi-head latent attention (MLA) compressed the memory a model needs for long conversations, and its very low API price pushed Chinese rivals into a price war in mid-2024. The same efficiency ideas, scaled up, produced the shock that came seven months later. Our context window history explains why that memory saving matters so much.

January 2025: The DeepSeek Shock

DeepSeek R1, released January 20, 2025, is the moment open weights went from "impressive" to "geopolitically significant." Built by a lab spun out of the Chinese hedge fund High-Flyer and led by Liang Wenfeng, R1 was a 671-billion-parameter MoE reasoning model, released under the permissive MIT license, that matched top proprietary reasoning models on math and code — and briefly rocketed DeepSeek's app to #1 on the US App Store.

How Much Did DeepSeek Cost to Train?

DeepSeek reported that the core training run cost under $6 million on roughly 2,000 Nvidia H800 GPUs over about two months. That figure covers the final run, not the full research and infrastructure bill, so it understates the true investment. Anthropic's CEO Dario Amodei argued that the savings fit the normal curve of falling training costs and were far smaller than the headlines implied. But even discounted, it reset expectations about what frontier-class training had to cost.

The exact numbers come from the DeepSeek-V3 technical report of December 2024, because R1 was trained on top of the V3 base model:

How R1 Learned to Reason

R1's real novelty was the recipe, which DeepSeek published in full. A first model, R1-Zero, learned to reason through reinforcement learning alone, with rule-based rewards for correct answers and the right format, and no human-written examples of reasoning. DeepSeek then added a small "cold start" set of long reasoning examples, more rounds of reinforcement learning with its GRPO method, and a final round of general tuning. It also released six smaller "distilled" models built on Qwen and Llama, so people without a data center could run an R1-style reasoner.

For background on why this mattered, read AI reasoning models explained and our wiki entries on model distillation and reasoning models.

Why Did DeepSeek R1 Crash Nvidia's Stock?

On Monday, January 27, 2025, a week after the release, DeepSeek's app passed ChatGPT at the top of the US App Store, and investors panicked. If a frontier reasoning model could be trained and served for a fraction of the expected cost, maybe the world needed fewer GPUs. Nvidia's shares fell about 17% that day, a loss of close to $600 billion in market value, the largest one-day loss for any US company at the time.

The fear faded within months. Cheaper models led to more AI use, not less, and demand for GPUs kept rising. The Nvidia and Jensen Huang history and our guide to how AI data centers work cover the hardware side.

Is DeepSeek Open Source?

Largely yes, as open weights. DeepSeek-R1 shipped in January 2025 with code and weights under the MIT license, and the V4 models kept MIT in 2026, which makes DeepSeek one of the most permissive frontier-class labs anywhere. Earlier models, including DeepSeek-V2 and the first DeepSeek-V3 release, used MIT for code but a separate DeepSeek model license with use restrictions, which drew the same "is it really open?" scrutiny Llama faced. DeepSeek moved V3 to MIT with the V3-0324 update in March 2025. The training data is not released, so "open weight" is still the precise term. The trajectory has been toward openness, and DeepSeek's low-cost, MIT-licensed models pushed the whole market's prices down. See DeepSeek vs ChatGPT for a direct comparison.

2025: Llama 4, Qwen3, Kimi K2 and the Year Open Models Learned to Act

2025 was the year open models stopped chasing chat quality and started chasing agent work: tool calls, long coding tasks and step-by-step reasoning. It was also the year the open lead passed, for most observers, from Meta to Chinese labs.

Llama 4: Meta's Last Llama (April 5, 2025)

Meta released Llama 4 Scout and Llama 4 Maverick on Saturday, April 5, 2025, as its first mixture-of-experts Llamas and its first natively multimodal ones. Scout has 109 billion total parameters, 17 billion active and 16 experts, with a context window of up to 10 million tokens. Maverick has about 400 billion total, the same 17 billion active, and 128 experts. A third model, Llama 4 Behemoth, with about 2 trillion parameters, was previewed as still in training and was never released.

The launch went badly. Meta had used a specially tuned "experimental" version of Maverick on the public LMArena leaderboard, which drew criticism once developers found the downloadable model behaved differently. Llama 4 did not become the default base model the way Llama 2 and 3 had. It was the last Llama release, and a year later Meta changed course entirely.

Qwen3: Hybrid Thinking Under Apache 2.0 (April 28, 2025)

Alibaba released Qwen3 on April 28, 2025: six dense models from 0.6B to 32B and two MoE models, 30B-A3B and the flagship 235B-A22B, all under Apache 2.0. The models trained on about 36 trillion tokens across 119 languages and dialects, and each one could either answer fast or think step by step. Qwen3-Coder, a 480-billion-parameter MoE for agentic coding, followed on July 22. By mid-2025 Qwen was the most common base for new fine-tunes on Hugging Face. The Qwen history tells the full story.

Kimi K2 and GLM-4.5: Trillion-Scale Agents (July 2025)

Moonshot AI released Kimi K2 on July 11, 2025: 1 trillion total parameters, 32 billion active, 384 experts, trained on 15.5 trillion tokens with the company's MuonClip optimizer. It was built for agent work, and its license was a modified MIT that asked only one thing of very large users: products with more than 100 million monthly active users or more than $20 million in monthly revenue must display "Kimi K2" in their interface. Commentators called it a second DeepSeek moment.

Two weeks later, on July 28, Zhipu AI (now Z.ai) released GLM-4.5, 355 billion total and 32 billion active parameters, under plain MIT. MiniMax had already shipped its 456B MiniMax-M1 in June with a 1-million-token context. By the end of July, the four strongest open-weight families were all Chinese. Compare two of them in Kimi vs DeepSeek and Kimi vs GLM.

The Rest of 2025: Thinking Models, Sparse Attention and OLMo 3

The second half of the year filled in the gaps:

• August 2025: DeepSeek V3.1 merged a thinking mode and a fast mode into one model. xAI published the weights of Grok 2.
• September 2025: DeepSeek-V3.2-Exp introduced DeepSeek Sparse Attention, which reads only the parts of a long context that matter for each token. GLM-4.6 followed on September 30.
• October 2025: MiniMax-M2 (about 230B total, 10B active) targeted coding agents at low cost.
• November 2025: Kimi K2 Thinking added long step-by-step reasoning with hundreds of tool calls. The Allen Institute for AI released OLMo 3, again with weights, code and data.
• December 2025: DeepSeek-V3.2 became a stable release on December 1, and Mistral Large 3 (675B total, 41B active) shipped under Apache 2.0.

gpt-oss: Why OpenAI Released Open Weights Again (August 2025)

On August 5, 2025, OpenAI released gpt-oss-120b and gpt-oss-20b under Apache 2.0, its first open-weight language models since GPT-2 in 2019. The gap between the two releases was almost six years. Both are mixture-of-experts reasoning models with adjustable reasoning effort, and both ship with weights already in the 4-bit MXFP4 format, so the larger model fits on a single 80 GB GPU.

The release changed the politics of the field. In 2019 OpenAI had been the lab that held weights back. In 2025 it was publishing them. The release came two weeks after the US government's AI Action Plan of July 2025 called for encouraging American open-weight models, at a time when the strongest open models came from Chinese labs. The OpenAI history and our explainer on what GPT is put gpt-oss in context.

2025–2026: China's Open-Weight Surge — Qwen, Kimi, GLM, DeepSeek V4

Through 2025 and into 2026, the center of gravity for open-weight releases shifted markedly toward Chinese labs, each shipping under permissive licenses:

The capstone landed in July 2026: Kimi K3, a 2.8-trillion-parameter open-weight model — the largest open-weight model to date — with native vision and a 1-million-token context window, released July 16 with full weights following on July 27. A model that big, released in the open, would have been unthinkable when GPT-2's 1.5B weights were considered too risky to publish six years earlier. (The full Moonshot story is worth its own read: Moonshot AI & Kimi history.)

2026: The Trillion-Parameter Open Era

In 2026 the largest open models passed one trillion, then two trillion parameters, and the release pace hit roughly one major open flagship a month. Three labs, Moonshot AI, Alibaba and DeepSeek, each published an open model above 1.5 trillion parameters, and every one of them carried a context window of about 1 million tokens.

Not every 2026 flagship was open. Alibaba's Qwen 3.7 Max and Plus (May and June 2026) stayed API-only, and Meta's Muse Spark launched closed in April. The pattern that emerged is two-tier: labs open a strong model, and sell a stronger or more convenient one as a service.

Kimi K3: The Largest Open-Weight Model (July 2026)

Kimi K3 is the largest open-weight model released so far: 2.8 trillion parameters (2.78 trillion exactly), of which about 104 billion, roughly 3.7%, run for each token. Moonshot AI launched the API on July 16, 2026 and published the full weights and technical report on July 27. The model card calls it "the world's first open 3T-class" model.

The architecture breaks with the plain transformer. Of its 93 layers, 69 use Kimi Delta Attention, a linear-attention design that keeps a fixed-size memory per sequence, and 24 use gated multi-head latent attention, in a three-to-one pattern. An MoE layer picks 16 of 896 routed experts plus 2 shared experts for each token. Kimi K3 has a 1,048,576-token context window and native vision, and it ships its weights in the 4-bit MXFP4 format, a checkpoint of about 1.56 TB.

Moonshot AI founder Zhilin Yang explained the scaling ideas behind the K2 line in his GTC 2026 keynote, which is the best primary source on how the lab thinks:

Kimi K3 also runs inside Taskade. See the Kimi K3 model page, all Moonshot AI models, and the TSK-1 build results for Kimi. In our TSK-1 benchmark, Kimi K3 built more than almost any model and finished later than all of them. Compare it with closed rivals in Kimi vs Claude and Kimi vs GPT.

Qwen3.8, DeepSeek V4 and GLM-5: The Other Giants

Kimi K3 was not alone at the top. DeepSeek V4 Pro (preview April 24, general release August 13) has about 1.6 trillion total and 49 billion active parameters, a 1-million-token context, and the MIT license. Qwen3.8-2.4T-A95B (August 12) was the first time Alibaba opened a Max-class flagship, with 2.4 trillion total and 95 billion active parameters. GLM-5 (February 11) scaled Z.ai's line from 355B to 744B total parameters under MIT, and GLM-5.3 followed in August. Browse the DeepSeek lab page, the Alibaba lab page and the Z.ai lab page, or read the DeepSeek V4 Pro model page.

Meta's U-Turn: From Llama to Muse Spark and Back

Meta spent 2023 and 2024 as the face of open weights, then closed its frontier work in 2025 and 2026, then opened again. The sequence is the clearest sign that "open versus closed" is now a business choice made model by model, not a lab's identity.

After Llama 4, Meta formed Meta Superintelligence Labs under Alexandr Wang, who joined from Scale AI after Meta invested $14.3 billion in that company. MSL's first model, Muse Spark, launched on April 8, 2026 as a closed model inside Meta AI. On October 15, 2026, Meta released Muse Spark Open: an 816-billion-parameter MoE with about 42 billion active parameters, 256 experts with 8 active per token, a 1-million-token context, and text, image, video and audio input. Meta says it runs on two 512 GB Mac Studios or eight H200 GPUs in its 4-bit version.

Its license is the 2026 pattern in one document. The Muse Model License reads like MIT, adds a usage policy and a "Built with Muse Spark Open" notice, and requires a separate agreement with Meta for any company that sells model access as a service with more than $20 million in yearly revenue.

Why Are Chinese Open-Source Models Winning? The Debate

This is the most-discussed — and most-contested — question in the field, so let's be precise about what's argued versus what's settled.

The popular thesis: US GPU export controls (restricting the best chips to Chinese buyers) pushed Chinese labs toward software-efficiency breakthroughs — mixture-of-experts, latent attention, and cheaper training recipes — because "when you can't throw more compute at a problem, you find smarter solutions" (Matt White's framing). Layer on a culture of publishing and a "borrow-and-build" chain — one lab's technique (like DeepSeek's GRPO reinforcement-learning method) gets adopted and improved by the next — and you get rapid, compounding open progress.

The dissent is serious and named. Analysts including Gregory Allen (CSIS) argue that motivation to localize is not the same as acceleration, and that some Chinese chip-independence efforts predate the controls. Chris McGuire (CFR) argues the US chip lead is actually widening. And Anthropic's Dario Amodei argues DeepSeek's efficiency was "an expected point on an ongoing cost-reduction curve," not a resource-constraint miracle.

A careful reading of the download data supports humility on both sides. By one 2026 analysis, Chinese models edged ahead of US models in the share of downloads of newly released models — but that measures a rolling 12-month window, and on an all-time basis US models still dominate downloads. The honest summary: Chinese labs drove much of the open-weight momentum in 2025–26, the technical contributions are real, and the geopolitical cause is genuinely debated. (And a note on attribution: the widely used Muon optimizer originated in the international open-source community, not with any single lab or country — cite ideas by their lineage, not a flag.)

The Parameter Race: From 1.5 Billion to 2.8 Trillion

The largest open-weight model grew from 1.5 billion parameters (GPT-2, November 2019) to 2.8 trillion (Kimi K3, July 2026), more than 1,800 times larger in under seven years. The growth came in two waves: dense models up to Llama 3.1 405B in 2024, then sparse mixture-of-experts models that kept the active part per token far smaller than the total.

The chart uses the largest model from each year: GPT-2 (2019), GPT-J-6B (2021), BLOOM (2022), Falcon 180B (2023), DeepSeek-V3 (December 2024), Kimi K2 (2025) and Kimi K3 (2026). It counts text-generating LLMs only. Google's 1.6-trillion-parameter Switch Transformer checkpoint, an encoder-decoder research model, is left out. The active line shows the turn: after 2024 the total keeps climbing while the compute per token stays near the size of a 2023 dense model. Our wiki entries on scaling laws and model training explain why labs kept adding parameters.

   GPT-2 (Nov 2019)                    KIMI K3 (Jul 2026)
   ─────────────────────────           ─────────────────────────────
   1.5B parameters, all active         2.8T parameters, ~104B active
   48 layers                           93 layers (69 KDA + 24 MLA)
   1,024-token context                 1,048,576-token context
   ~40 GB of web text                  Text, images and video
   Dense transformer                   896 experts, 16 + 2 per token
   ~3 GB file at 16-bit                ~1.56 TB file at 4-bit
   Runs on one 2019 GPU                Needs a full 8x B300 node
   Held back for 9 months              Weights out 11 days after API
   Modified MIT license                Kimi K3 License (MIT + MaaS)

Open vs Closed: How Big Is the Gap in 2026?

The gap has never been smaller. On public benchmarks, the best open-weight models trail the top proprietary systems by a small and shrinking margin — often described in 2026 as a matter of a few months rather than the year-plus gap of 2023. Independent reviewers who tested Kimi K3 at launch placed it at or near frontier level for agentic coding, and one summed up the moment bluntly: "Open-source AI is no longer a few months behind the closed frontier models."

Two honest caveats keep this from becoming hype. First, the very hardest reasoning tasks still tend to favor the top proprietary models. Second, capability is not reliability — open frontier models can be slower, more token-hungry, and flakier on long runs than their polished closed rivals. The gap in raw capability is nearly closed; the gap in dependability is still real, and it's the reason AI-generated apps break more often than their demos suggest.

The Best Open-Source LLMs to Use Right Now

If you want a ranked answer to "which open model should I use in 2026," we keep a dedicated, continuously-updated guide: the 10 best open-source LLMs. The short version by job:

• Coding: Qwen3-Coder and the Kimi K2/K3 line lead agentic coding.
• Reasoning and math: DeepSeek V4 and the R1 lineage are the value picks.
• Long context: Kimi (1M tokens) and Llama 4 Scout (very long windows).
• Running locally: small Gemma, Qwen, and Llama variants fit a single GPU.

Benchmarks age fast, so we also test models on real app builds. Our TSK-1 benchmark write-up, the history of AI benchmarks and our guide to the best model for building apps explain the method.

The larger point: you rarely want just one. The most effective setup keeps several models available and routes each task to the best one — a strategy we break down in AI agent cost optimization.

To see which open models build a working app, the TSK-1 Benchmark sends the same app request to each model and records whether the app worked: DeepSeek, Kimi, Qwen and GLM each have a results page.

For the full story of two of these labs, read the DeepSeek history and the Qwen history.

Open-Source LLM License History: Five Eras in Seven Years

Open-model licenses went through five eras: withheld weights (2019), research-only licenses (2022 to early 2023), community licenses with a user threshold (Llama 2, 2023), plain Apache 2.0 and MIT (Mistral, Qwen, DeepSeek-R1, 2023 to 2025), and in 2026, MIT-style licenses with a clause for large model-as-a-service businesses.

The 2026 "MaaS clause" is new enough that most license guides miss it. The licenses read like MIT for nearly everyone: you can download, modify, fine-tune, sell products and build apps. The extra terms bite only when a company resells raw model access through an API and earns above a revenue line. We read each license file on Hugging Face:

Every license above exempts internal use, and the Kimi, Muse and GLM licenses define model-as-a-service so that it does not cover an end-user product that only embeds the model in its own features. Read the license file of the exact version you download, because terms differ within one family. Qwen3.8-27B, for example, is plain Apache 2.0, while the 2.4T flagship is not.

Open-Source LLM License Cheat-Sheet

Licenses are where "open" gets real. Here's the current lay of the land (always verify the specific version's model card — licenses change within families in both directions):

   MODEL FAMILY        LICENSE               COMMERCIAL USE
   ──────────────────────────────────────────────────────────
   Qwen (most sizes)   Apache 2.0            Yes, unrestricted
   Mistral / Mixtral   Apache 2.0            Yes, unrestricted
   gpt-oss (OpenAI)    Apache 2.0            Yes, unrestricted
   Gemma 4 (Google)    Apache 2.0            Yes, unrestricted
   DeepSeek (R1, V4)   MIT                   Yes, unrestricted
   GLM-5 (Z.ai)        MIT                   Yes, unrestricted
   Kimi (K2 family)    Modified MIT          Yes; attribution above
                                             100M MAU / $20M/mo rev
   Kimi K3             Kimi K3 License       Yes; deal needed for
                                             API resellers > $20M/yr
   Muse Spark Open     Muse Model License    Yes; notice + deal for
                                             API resellers > $20M/yr
   Qwen3.8 flagship    Custom Qwen license   Yes; deal for API
                                             resellers > $50M/yr
   Llama               Llama Community       Yes; restricted above
                                             700M MAU + use policy
   BLOOM               BigScience RAIL 1.0   Yes; use restrictions
   ──────────────────────────────────────────────────────────
   Rule of thumb: Apache 2.0 and MIT = the freest. Everything
   else has a clause worth reading before you build on it.

The Hardware Story: From Laptops to Trillion-Parameter Racks

Open weights only matter if someone can run them, so the hardware story runs in parallel with the model story. Between 2023 and 2026 two things happened at once: software tricks shrank mid-size models enough to run on a laptop, and the largest open models grew so big that only a full rack of data-center GPUs can hold them.

How Did Open Models Get Small Enough for a Laptop?

Quantization stores each parameter in fewer bits. A model trained at 16-bit precision uses 2 bytes per parameter; at 4-bit it uses half a byte, a quarter of the memory, with a small loss in quality. Three open projects made this normal:

How Much Memory Does an Open Model Need?

The quick rule is bytes per parameter times parameters, plus room for the context cache. The numbers below are for the weights alone.

   MEMORY FOR THE WEIGHTS (approximate)
   ───────────────────────────────────────────────────────────────
   Precision      Bytes/param   7B model   70B model   2.8T model
   ───────────────────────────────────────────────────────────────
   16-bit (BF16)  2.0           14 GB      140 GB      5.6 TB
   8-bit          1.0            7 GB       70 GB      2.8 TB
   4-bit          0.5          3.5 GB       35 GB     ~1.5 TB
   ───────────────────────────────────────────────────────────────
   + context (KV) cache: grows with every token in the prompt
   Laptop with 16-32 GB unified memory .... 7B to 14B at 4-bit
   One 80 GB data-center GPU .............. 70B at 4-bit, gpt-oss-120b
   8x B300 node (~2.3 TB of HBM) .......... Kimi K3 in native MXFP4
   8x H100 node (640 GB of HBM) ........... too small for Kimi K3

   CAN I RUN THIS OPEN MODEL MYSELF?
   │
   ├─ Under ~14B parameters?
   │     └─ YES → laptop or one consumer GPU (llama.cpp, Ollama, MLX)
   │
   ├─ 30B to 120B (or a small-active MoE)?
   │     └─ YES → one 24-80 GB GPU at 4-bit, or a 64-128 GB Mac
   │
   ├─ 400B to 1T?
   │     └─ MAYBE → a multi-GPU server, or rent one by the hour
   │
   └─ Over 1T (Kimi K3, Qwen3.8 flagship, DeepSeek V4 Pro)?
         └─ NO for most teams → use a hosted provider or a platform

Why Total Parameters, Not Active Ones, Set the Hardware Bill

A mixture-of-experts model runs only a few experts per token, so people often assume it needs only a little memory. It does not. When a server handles many users at once, almost every expert is in use at some instant, and swapping experts in and out of memory would wreck response times. So the GPUs must hold all the weights. That is why Kimi K3, with about 104 billion active parameters, still needs a node of eight Nvidia B300 GPUs with 288 GB of memory each, and does not fit on eight H100s with 80 GB each.

Hosts split such a model across GPUs in two main ways. Tensor parallelism cuts every layer into slices across eight GPUs, which gives the best latency for one user. Expert parallelism keeps attention split but places whole experts on specific GPUs, which serves more users per node. In October 2026, independent measurements showed the same Kimi K3 weights running roughly ten times faster on the fastest provider than on the slowest, because of these choices and the hardware behind them. Baseten's talk walks through it step by step:

For the deeper hardware layer, see our wiki entries on the GPU, high-bandwidth memory, memory bandwidth, inference and tokens per second, plus our guide to reducing LLM costs.

How to Use Open Models Without the Setup Headache

Here's the practical payoff of this whole history. You don't have to pick one open model, and you certainly don't have to rent GPUs to use them. The lesson of the last six years — GPT-2's caution, Llama's ecosystem, DeepSeek's cost shock, Kimi K3 in the open — is that the frontier is becoming a commodity input. The durable advantage is the layer that lets you swap models freely.

That's how Taskade Genesis works. It gives you frontier models from top AI labs, including open-weight models, selectable per agent, with Auto picking a sensible default. Route cheap high-volume work to an open-weight model and hard reasoning to a premium one, all in one workspace, no lock-in.

Here is exactly where the open models in this history are available inside Taskade:

An agent chat on your own key uses no Taskade credits. See every option on the models directory, the key providers page and the model docs for Kimi, DeepSeek and GLM. The walkthrough for adding a key is in AI Model Keys on Learn Taskade. For a wider view, compare the no-code builders that run open-weight models and the AI builders that accept your own API key.

Every AI agent gets built-in tools, persistent memory, and 100+ bidirectional integrations. Your projects become the memory, your agents the intelligence, your automations the execution:

Build your own AI app from a prompt → — describe it once, and Taskade Genesis builds it as living software you can run on whichever models you like.

What's Next for Open-Source LLMs

Three trends are already visible. Trillion-parameter open models are becoming normal — Kimi K3 (2.8T) and DeepSeek V4-Pro (~1.6T) put frontier-scale weights in the open. The gap keeps shrinking toward a few months, which changes the economics of building on open models. And attention is moving from raw chat quality to agentic and tool-use capability — models that can run long, multi-step tasks — plus on-device inference for privacy. The through-line: openness compounds, and the community keeps closing the distance.

Four more signals from the second half of 2026 point to what comes next:

1. Licenses are converging. Kimi K3, Qwen3.8, GLM-5.3 and Muse Spark Open all landed on the same shape: MIT-style freedom for builders, plus one clause for the largest API resellers. Expect more labs to copy it.
2. US labs are back in the open. gpt-oss, Gemma 4 under Apache 2.0, NVIDIA's Nemotron line and Muse Spark Open mean the open frontier is no longer only a Chinese story.
3. Architectures are diverging from the plain transformer. Kimi K3's linear attention, DeepSeek's sparse attention and MiniMax's sparse attention all attack the cost of million-token contexts.
4. 4-bit is the new default. gpt-oss and Kimi K3 shipped native 4-bit weights, so "open" increasingly means "open and already compressed to run."

For where agents built on these models are heading, read LLM vs RAG vs AI agents vs agentic AI and what AI agents are.

▲ ■ ●  The labs build the models; you build the apps that run on them. Don't just read the open-source timeline — clone a living model tracker, point it at any lab, and own a research workspace in minutes. Memory feeds Intelligence, Intelligence triggers Execution. Clone the live tracker → or build your own from a prompt →.

🔗 Related Reading

Models & labs:

• The 10 best open-source LLMs, ranked
• Moonshot AI & Kimi history
• Anthropic & Claude history
• OpenAI & ChatGPT history
• What is GPT? From GPT-1 to GPT-6
• Google Gemini history
• NVIDIA & Jensen Huang history
• DeepSeek history
• Alibaba Qwen history
• The history of OpenAI Codex
• Claude models explained

Use open models:

• No-code builders that run open-weight models
• AI builders that accept your own API key
• The best AI agent platforms
• The best AI agent builders
• The best free AI app builders
• The best AI app builders
• Qwen vs DeepSeek

How it works:

• The history of the context window
• AI inference explained
• How large language models work
• What are reasoning models?
• What are world models?
• What is mechanistic interpretability?
• The history of computing

🐑 Before you go — you don't have to choose one model. Inside Taskade Genesis you get:

• Frontier and open-weight models from top AI labs, selectable per agent
• built-in tools and persistent memory on every AI agent
• 100+ bidirectional integrations to automate the busywork
• Auto model routing — the right model per task, no lock-in

Start free → · Explore ready-made AI apps →

💬 Frequently Asked Questions About Open-Source LLMs