Chips, compute and data centres
GPU
In short
A GPU, or graphics processing unit, is a computer chip with thousands of small cores that do many simple calculations at once. Built for video games, it is now the main chip used to train and run AI models.
- Also called
- Graphics processing unit, Graphics card, AI GPU
- Topic
- Chips, compute and data centres
- Level
- Beginner
A normal processor (CPU) is built to run a few tasks very quickly, one step after another. A GPU makes the opposite trade. Nvidia's own programming guide says a CPU can run a few tens of threads in parallel, while a GPU is designed to run thousands, accepting slower single threads for much greater total throughput1. The UK competition regulator describes a GPU the same way: a chip with many small, specialised cores that run in parallel2.
That suits AI. Training a neural network is mostly multiplying large grids of numbers, and each multiplication can be done side by side. In 2012 the AlexNet image model, a turning point for deep learning, was trained using "a very efficient GPU implementation"3.
Graphics chips for gaming and AI chips for data centres share the idea but differ in scale. Nvidia's DGX B200 server packs eight Blackwell GPUs with 1,440 GB of high-bandwidth memory between them and draws up to about 14.3 kW4. A gaming card in a home PC has a small fraction of that.
A GPU is not the only option. The main alternative is a custom chip, or ASIC, hardwired for a specific job2, such as Google's TPU. GPUs are more general-purpose, so they can run a wider range of software.
Example
When a company like Meta trains a large language model, it does so on thousands of GPUs at once. Meta's Llama 3 405B model was trained on up to 16,000 Nvidia H100 GPUs5.
Why it matters
GPUs are the scarce, expensive resource behind the AI boom. Who can buy them, and how many, shapes which companies and countries can build frontier AI.
Questions people ask
Why are GPUs used for AI?
Because AI models are mostly huge numbers of simple multiplications that can run side by side. A GPU is designed to run thousands of threads in parallel, so it gets through that work far faster than a CPU1.
What is the difference between a CPU and a GPU?
A CPU devotes more of its transistors to caching and control so it can run a sequence of steps quickly. A GPU devotes more to arithmetic units so it can run thousands of calculations at once1.
Can I run AI on my own GPU?
Yes, smaller models can run on a gaming graphics card or a laptop chip. The largest models need data centre GPUs with far more memory, such as servers holding eight GPUs and 1,440 GB of memory4.
Related terms
Sources
- Nvidia, CUDA Programming Guide, Introduction (benefits of using GPUs)
- Competition and Markets Authority (2025), Cloud services market investigation, final decision glossary
- Krizhevsky, Sutskever and Hinton (2012), ImageNet Classification with Deep Convolutional Neural Networks, NeurIPS 25
- Nvidia, DGX B200 product specifications
- Grattafiori et al. (2024), The Llama 3 Herd of Models, arXiv:2407.21783, section 3.3.1
- Nvidia, Corporate timeline