TPU

In short

A TPU, or Tensor Processing Unit, is Google's own chip for AI. It is a custom-designed chip that does the matrix maths inside neural networks very efficiently, and Google runs its Gemini models on it.

Also called: Tensor Processing Unit, Cloud TPU, Google TPU
Topic: Chips, compute and data centres
Level: Some background helps
First described: Google had TPUs in its data centres from 2015. Norman Jouppi and colleagues described the first one publicly in April 2017, reporting it was about 15 to 30 times faster at inference than contemporary GPUs and CPUs⁵.

Google describes TPUs as application-specific integrated circuits (ASICs), meaning chips custom-built for one kind of job, in this case speeding up machine learning¹. A GPU is more general-purpose. A TPU gives up some of that flexibility to do matrix maths faster and with less power.

The core of each chip is a matrix-multiply unit. It is laid out as a systolic array: thousands of multiply-accumulate circuits wired directly to each other, so numbers flow from one to the next without trips back to memory². Multiplications take inputs in a compact number format called bfloat16, while running totals are kept in 32-bit numbers².

TPUs are joined into pods over a dedicated network, and customers rent a slice of a pod². Google's seventh generation, Ironwood, announced in April 2025, scales to 9,216 liquid-cooled chips with 192 GB of high-bandwidth memory on each³.

Google says Gemini 2.5 and AlphaFold run on TPUs³. Outside Google, Anthropic announced in October 2025 that it would use up to one million TPUs, alongside Amazon's Trainium chips and Nvidia GPUs⁴.

TPUs are not suited to everything. Google's own guidance says they are a poor fit for work that needs high-precision arithmetic¹.

Example

When you ask Gemini a question, the answer is likely computed on TPUs in a Google data centre rather than on Nvidia GPUs³.

Why it matters

TPUs are the best-known alternative to Nvidia's GPUs. They give Google, and anyone renting them, a way to run large AI models without depending on a single chip supplier.

Questions people ask

Is a TPU better than a GPU?

It depends on the job. Google's guidance says TPUs suit models dominated by matrix maths that train for weeks or months, while GPUs suit models with many custom operations. Neither is better for everything¹.

Can I buy a TPU?

Not as a card for your own computer. Google offers TPUs to rent through Google Cloud, in groups called slices¹².

What software runs on TPUs?

Google lists TensorFlow, PyTorch and JAX. A compiler called XLA turns the model into instructions the TPU can run¹.

Related terms

Tools that use it

Google GeminiGoogle's chatbot, built into Android, Gmail and Docs, with a generous free plan.

Sources

Back to the glossary