Reinforcement learning

In short

Reinforcement learning is a way of training a system by letting it try actions and scoring the results with a number called a reward. Nobody shows it the correct action. It has to find choices that bring more reward over time.

Also called: RL
Topic: How models are trained
Level: Some background helps
First described: Sutton and Barto write that the computing use of the phrase follows engineering and AI work influenced mostly by Marvin Minsky's 1961 paper 'Steps Toward Artificial Intelligence'. Alan Turing sketched a pleasure-pain system along the same lines in 1948. They date the modern field to the late 1980s, when trial-and-error learning was joined to optimal control¹.

Reinforcement learning is learning what to do: how to map a situation to an action so as to maximise a numerical reward. Sutton and Barto, whose textbook defines the field, stress two features. The learner is not told which action to take, and must find out by trying. And an action can change not only the immediate reward but the next situation, and through that every reward that follows¹.

NIST's 2025 glossary says the same thing in shorter form: a model learns to optimise its behaviour according to a reward function by interacting with an environment and getting feedback from it². Sutton and Barto also separate it from the other two standard paradigms. Supervised learning is given the correct action for each example. Unsupervised learning looks for structure in unlabelled data. Reinforcement learning does neither. It maximises a reward¹.

A difficulty that does not arise in those other two, at least in their pure form, is when to repeat an action that has already paid off and when to try something new. Sutton and Barto call this the trade-off between exploitation and exploration¹.

It shows up in games and, later, in chatbots. In 2013 Mnih and colleagues at DeepMind trained a neural network by reinforcement learning on the raw pixels of seven Atari 2600 games, with no change of design between games. It beat earlier methods on six of the games and a human expert on three³. The last stage of OpenAI's InstructGPT training was also reinforcement learning: a reward model trained on human rankings scored each reply, and the language model was updated to raise that score⁴. The ChatGPT model OpenAI introduced in November 2022 used that same step⁵.

Example

A program can learn a video game from the picture on the screen, without a list of correct button presses. DeepMind's 2013 system did that for seven Atari 2600 games, and beat a human expert on three of them³.

Why it matters

It is how a system learns when the right move is not written down in advance, only a score for how things turned out. That covers DeepMind's Atari player³ and the last training stage of InstructGPT⁴ and of the November 2022 ChatGPT model⁵.

Questions people ask

Is reinforcement learning the same as supervised learning?

No. Supervised learning is given the correct action or category for each example. Reinforcement learning is not. The system tries actions and is told only how much reward followed¹.

What is a reward in reinforcement learning?

A number that scores the result of an action. Sutton and Barto define the learner's job as choosing actions so as to maximise that number over time, including rewards that arrive only later¹. NIST calls the rule that produces the number a reward function².

Is reinforcement learning from human feedback the same as reinforcement learning?

It is a form of it. In ordinary reinforcement learning a programmer writes the reward. In the InstructGPT setup, people rank model replies, a reward model learns to imitate those rankings, and reinforcement learning then pushes the language model toward higher scores from that model⁴.

Related terms

Tools that use it

ChatGPTOpenAI's general chatbot, the most used AI app in the UK and worldwide.

Sources

Back to the glossary