Neural network

In short

A neural network is a computer model loosely inspired by the brain. It is made of many simple units, called nodes, linked by connections whose strengths are adjusted during training until the network gives useful answers.

Also called: Artificial neural network, ANN, Neural net
Topic: The basics
Level: Beginner
First described: Warren McCulloch and Walter Pitts, December 1943, "A logical calculus of the ideas immanent in nervous activity", which described networks of simplified neurons in logical terms³.

In an artificial neural network, nodes stand in for brain cells and the connections between them stand in for synapses. Those connections "can be made stronger or weaker", and adjusting them using examples is called training¹.

Data flows in at one end, passes through layers of nodes, and comes out as an answer, such as a label for a photo. Each connection carries a number, often called a weight. During training, the backpropagation algorithm works out how each of these internal parameters should change².

The idea is older than the personal computer. In 1943 Warren McCulloch and Walter Pitts showed that networks of simplified all-or-none neurons could be described with propositional logic³. In 1958 Frank Rosenblatt published the perceptron, a probabilistic model of how the brain might store and organise information⁴.

Interest faded at the end of the 1960s after discouraging theoretical results, then revived in the 1980s¹. Many researchers lost interest again in the 1990s before a new run of results from 2006¹. Deep networks with many layers now underpin speech and image recognition², and in 2024 John Hopfield and Geoffrey Hinton won the Nobel Prize in Physics for foundational work on them⁵.

Despite the name, a neural network is not a digital brain. The brain was the original inspiration¹, but an artificial network is a mathematical model that runs on ordinary computer chips.

Example

A network that reads handwritten numbers takes each image as a grid of pixel values, passes it through its layers, and outputs the most likely digit. Hinton's 2006 deep network was tested on exactly this task with handwritten digit images⁶.

Why it matters

Neural networks are the building block of nearly all modern AI, including the large language models behind chatbots.

Questions people ask

Is a neural network like a human brain?

Only loosely. Its nodes and connections were modelled on neurons and synapses¹, but it is a mathematical model running on computer chips, not living tissue.

Who invented neural networks?

Warren McCulloch and Walter Pitts described networks of simplified neurons in 1943³, and Frank Rosenblatt published the perceptron in 1958⁴. John Hopfield and Geoffrey Hinton won the 2024 Nobel Prize in Physics for later foundational work⁵.

What is the difference between a neural network and deep learning?

Deep learning means using neural networks with many layers, each building a more abstract representation of the data². All deep learning uses neural networks, but not every neural network is deep.

Related terms

Sources

Back to the glossary