Turing test

In short

The Turing test is Alan Turing's imitation game: a person tries to tell a machine from a human using written conversation. Turing offered it instead of asking whether machines can think, and he argued that it does not require evidence of felt consciousness.

Also called: Imitation game, Turing's imitation game
Topic: Testing and benchmarks
Level: Beginner
First described: Alan Turing introduced the imitation game in Computing Machinery and Intelligence, published in Mind in October 1950¹.

In his 1950 paper Computing Machinery and Intelligence, Alan Turing replaces the question "Can machines think?" with an imitation game. An interrogator, kept apart from the others, uses written or typewritten questions to tell a man and a woman apart. Turing asks what happens if a machine takes the man's part. He later asks whether a programmed computer could play that deceiving part while a man takes the other. The interrogator cannot see, touch or hear the players¹.

Turing calls "Can machines think?" too meaningless to deserve discussion. He does not define one pass mark. He predicts that in about fifty years an average interrogator might have no more than a 70 per cent chance of a correct identification after five minutes. Against the objection that a machine must feel emotions to count as thinking, he argues that demanding felt consciousness leads to solipsism, and that people who drop that view should accept the imitation game¹.

A 2024 preregistered study used a two-player version, which the authors say differs from Turing's three-person game. Each interrogator had one text conversation of up to five minutes with either a human or an AI, then judged which it was. With a prompt telling it to sound like a casual young person, GPT-4 was judged human 54% of the time, not significantly different from chance. Human witnesses were judged human 67% of the time, and the simple ELIZA program 22%. The authors treat the GPT-4 result as a pass of that test².

The same paper says a result like this is unlikely to be necessary or sufficient evidence of intelligence. It is one prompt, one time limit and one design. Jones and Bergen also write that chatbots are already used for jobs such as customer service, so people already have to ask whether the other party is a person².

Example

In the 2024 study, a person chatted by text for up to five minutes and then said whether the other party was human. GPT-4 was judged human in 54% of its games, a rate the authors count as passing because it was not distinguishable from chance².

Why it matters

Turing offered the imitation game in place of asking whether machines think, and argued that the test does not require evidence of felt consciousness¹. A controlled study still treats a pass as weak evidence about intelligence, and as evidence about whether a person can tell a machine from a human in a short text chat².

Questions people ask

What does the Turing test test?

Turing's imitation game asks whether an interrogator can tell a machine from a person by written conversation. He called "Can machines think?" too meaningless to deserve discussion, and he argued against requiring evidence that the machine feels emotions¹.

Has ChatGPT passed the Turing test?

A 2024 study tested GPT-4, through an API, in one five-minute, two-player text chat with a set prompt. Interrogators judged it human 54% of the time, which the authors count as a pass of that test. Human witnesses in the same study were judged human 67% of the time. That is not a result for every chatbot or every setup².

Is the Turing test still used?

Researchers have kept running Turing-style tests. Jones and Bergen's 2024 experiment is a recent controlled example, and they describe a long line of earlier attempts. They also say the test is unlikely to be necessary or sufficient evidence of intelligence².

Related terms

Tools that use it

Sources

Back to the glossary