Foundation model

In short

A foundation model is a large AI model trained on huge amounts of broad data so it can be adapted to many different jobs, such as chat, coding or image making. GPT, Claude and Gemini are built on them.

Also called: Foundation models, FM
Topic: The basics
Level: Some background helps
First described: Named by Bommasani and more than 100 co-authors at Stanford's Center for Research on Foundation Models in August 2021, in "On the Opportunities and Risks of Foundation Models"¹.

Older AI models were usually built for one task, such as spotting spam. A foundation model is trained once, on a very wide spread of data, and then adapted for many tasks afterwards.

The term comes from a 2021 Stanford paper. It described models such as BERT, DALL-E and GPT-3 as "trained on broad data at scale" and "adaptable to a wide range of downstream tasks"¹. The authors chose "foundation" to stress that these models are "critically central yet incomplete": much is built on top of them, and any flaw they carry is passed on to everything built on them¹.

In practice, a company trains one big model and then adapts it, for example by fine-tuning it on extra examples, or simply by giving it instructions. The UK competition regulator, the CMA, describes foundation models as "large, machine learning models trained on vast amounts of data"².

A foundation model is not the app you use. ChatGPT, Claude and Gemini are products, and the large language models underneath them are the foundation models³. Not all foundation models handle text: DALL-E, which makes images, was one of the original examples¹.

EU law uses a different label. The EU AI Act does not say "foundation model"; it regulates "general-purpose AI models", defined as models that display significant generality and can competently perform a wide range of distinct tasks⁴.

Example

A bank might take a general foundation model and adapt it to answer customer questions about its own accounts, rather than building a chatbot from scratch.

Why it matters

A small number of these models sit underneath a huge number of AI products. Who builds them, and how well, shapes the quality, price and safety of almost every AI tool people use.

Questions people ask

Is a foundation model the same as a large language model?

Not exactly. Most large language models are foundation models, but the term is wider. It also covers models for images and other data. The paper that coined it gave DALL-E, an image generator, as an example alongside GPT-3¹.

Who coined the term foundation model?

Researchers at Stanford University's Center for Research on Foundation Models, in a long report led by Rishi Bommasani and posted in August 2021¹.

How many foundation models are there?

Many, and the number keeps growing. In September 2023 the UK's Competition and Markets Authority estimated that around 160 had been developed and released since OpenAI released the first public one in 2018². No up-to-date official count is available.

What does the EU AI Act call foundation models?

General-purpose AI models. The Act defines them as AI models that show significant generality, can competently perform a wide range of distinct tasks and can be built into many downstream systems⁴.

Related terms

Tools that use it

Sources

Back to the glossary