AI misinformation
In short
AI misinformation is false, inaccurate or misleading content that generative AI makes easier to produce or spread without an intent to deceive. When someone uses those tools on purpose to deceive or harm, NIST calls that disinformation.
- Also called
- AI-generated misinformation
- Topic
- Safety, ethics and risk
- Level
- Beginner
NIST says generative AI can ease the unintentional production or dissemination of false, inaccurate or misleading content, which it calls misinformation, particularly when that content comes from confabulations.1 The same section says these systems can also ease the deliberate production or dissemination of false or misleading information, which it calls disinformation, where an actor has the explicit intent to deceive or cause harm.1 Even very subtle changes to text or images can manipulate human and machine perception.1
NIST says both misinformation and disinformation may erode public trust in true or valid evidence.1 Its example is a synthetic image of a Pentagon blast that went viral and briefly caused a drop in the stock market.1 Generative models can also help actors make imagery and propaganda that is not photorealistic, but that could still gain reach on social media, and fraudulent content intended to impersonate other people.1
A 2023 UNESCO guidance says that, despite fluent and impressive output, generative AI cannot be trusted to be accurate.2 It quotes ChatGPT's provider: answers that sound reasonable cannot be relied upon to be accurate.2 It says the errors will most often go unnoticed unless the user already has a solid knowledge of the topic.2
NIST defines confabulation separately, as a system confidently presenting erroneous or false content.1 Misinformation in this profile is the wider problem of producing or spreading false content, especially when a confabulation is the source. A deepfake, in the same section, is one kind of synthetic audiovisual content or photorealistic image.1
Example
NIST's example is a synthetic image of a blast at the Pentagon. The image went viral and briefly caused a drop in the stock market.1
Why it matters
NIST says misinformation and disinformation facilitated by generative AI may erode public trust in true or valid evidence, and UNESCO says fluent generative output cannot be trusted to be accurate.12
Questions people ask
What is the difference between misinformation and disinformation?
In NIST's generative-AI profile, misinformation is false, inaccurate or misleading content produced or spread unintentionally.1 Disinformation is false or misleading information produced or spread on purpose, where the actor's explicit intent is to deceive or cause harm.1
Is a wrong chatbot answer misinformation?
NIST counts unintentional production of false content as misinformation, particularly when it stems from confabulation, which is a confident false answer.1 A wrong reply can therefore be both. It is disinformation only where an actor has an explicit intent to deceive or cause harm.1
How can I check something generative AI produced?
UNESCO says generative AI cannot be trusted to be accurate, and that errors most often go unnoticed unless the reader already knows the topic well.2 NIST adds that this material may erode trust in valid evidence, so a fluent answer is not itself a check.1