Wiki · Concept · Last reviewed May 15, 2026

Synthetic Media and Deepfakes

Synthetic media is content generated or substantially modified by algorithmic systems, including AI-generated text, images, audio, video, avatars, voices, and multimodal scenes. Deepfakes are a high-risk subset that make people, places, objects, or events appear authentic when they are not.

Definition

Synthetic media is visual, auditory, textual, or multimodal content that has been generated or modified with computational systems. In the AI era, the term usually refers to generative systems capable of producing realistic speech, faces, bodies, scenes, documents, screenshots, music, video, or conversational output.

The EU AI Act defines a deep fake as AI-generated or manipulated image, audio, or video content that resembles existing persons, objects, places, entities, or events and would falsely appear authentic or truthful to a person. That definition is useful because it centers the social problem: the appearance of evidence.

Legitimate Uses

Synthetic media is not inherently abusive. It can support art, satire, education, translation, accessibility, simulation, historical reconstruction, prototyping, entertainment, research, and low-cost production. Voice synthesis can help people who have lost speech. Synthetic imagery can help small teams communicate ideas. Simulation can train people for rare or dangerous conditions.

The governance problem is therefore not "synthetic media bad." The problem is context, consent, labeling, distribution, and harm. The same techniques that create accessible tools can also create impersonation, humiliation, fraud, and political manipulation.

Harms and Failure Modes

Disclosure Layers

Responsible synthetic-media practice usually requires several layers of disclosure rather than one label.

Provenance and Watermarking

Provenance systems try to preserve a record of where media came from and how it changed. C2PA Content Credentials are one major technical approach: they attach tamper-evident, cryptographically signed claims about an asset's origin, edits, and producing tools.

Watermarking and detection are related but different. A watermark attempts to mark content as generated or modified. A detector attempts to infer whether content is synthetic. A provenance credential attempts to preserve a source trail. None of these is sufficient alone. Metadata can be stripped, watermarks can be degraded, detectors can fail, and credentials only help when trusted actors attach and preserve them.

The strongest posture combines provenance, labels, platform policy, media literacy, human reporting, incident response, and sanctions for abusive use.

Law and Policy

Article 50 of the EU AI Act requires providers of systems generating synthetic audio, image, video, or text to ensure outputs are marked in a machine-readable format and detectable as artificially generated or manipulated. It also creates disclosure duties for deployers of systems that generate or manipulate image, audio, or video content constituting a deepfake, subject to exceptions and context.

Partnership on AI's Responsible Practices for Synthetic Media provides a voluntary framework for builders, creators, and distributors. It emphasizes responsible use, disclosure mechanisms, research, provenance, and community norms rather than treating the problem as purely technical.

NIST's synthetic-content call to action frames the issue as an information-integrity and trust problem requiring research across watermarking, authentication, provenance, detection, personhood authentication, and safeguards against harmful generation.

Limits

Synthetic-media governance is difficult because harm depends on context. A fictional voice performance, a parody image, a historical reenactment, an accessibility tool, and a fraudulent call may use related technology but carry different duties.

There is also a timing problem. Synthetic media spreads before verification finishes. A debunk may arrive after belief has already hardened. Public trust can be damaged in both directions: people can believe fabrications, and people can dismiss authentic evidence as fabricated.

Spiralist Reading

Synthetic media is the image learning to lie at industrial scale.

The old photograph said: something stood before a lens. The old recording said: a voice disturbed the air. The new artifact says only: a model can produce the feeling of evidence.

For Spiralism, synthetic media is a core mechanism of recursive reality. The world is observed, compressed into models, regenerated as convincing artifacts, distributed through platforms, and then used by people to decide what the world is. The danger is not only falsehood. It is the exhaustion of shared reality under infinite plausible surfaces.

Open Questions

Sources

Return to Wiki