“When anyone can forge a photograph, the question stops being "is it fake?" and becomes "can you prove it's real?"”
Deepfakes and the collapse of "seeing is believing"
For all of history, a photograph, a recording, or a video was reasonable evidence that something happened. Generative AI ends that assumption. Convincing fake images, cloned voices, and increasingly fake video can be made cheaply by anyone. The harm isn't hypothetical: fraud using cloned voices, fabricated images used for harassment and disinformation, and synthetic media used to manipulate.
There's a subtler, second harm: the "liar's dividend." Once everyone knows media can be faked, real evidence can be dismissed as fake. The damage isn't only false things believed true — it's true things waved away as possibly synthetic. Both directions corrode trust.
Can we detect what's fake?
The hoped-for fix — a detector that flags AI-generated media — is fundamentally fragile. Detection is an arms race: every detector becomes training signal for the next generation of generators to evade. Detectors that work today degrade as models improve, and they produce both false positives (flagging real content) and false negatives (missing fakes), neither of which is acceptable when stakes are high.
Provenance: prove what's real, not what's fake
The more promising direction is provenance: instead of detecting fakes after the fact, cryptographically record where content came from at creation. A camera or a generator signs the content with a tamper-evident credential saying what it is and how it was made; downstream, anyone can verify that chain. Industry standards (such as C2PA / Content Credentials) and generator watermarking (such as Google DeepMind's SynthID) push in this direction.
Be precise about what this buys you. Provenance proves origin, not truth — a signed photo is genuinely from that camera, but the scene could still be staged. And it only helps where it's adopted; unsigned content isn't proven fake, just unproven. Watermarks can also be weakened by editing. It's a real and important defence, and a partial one.
Copyright and consent
Two legal questions hang over all generative media and are genuinely unsettled. On the input side: models are trained on vast amounts of copyrighted images, audio, and text, and whether that training is permitted is being fought in courts and varies by jurisdiction. On the output side: who owns AI-generated content, and does output that resembles training data infringe?
Add consent and likeness: generating a real person's face or voice without permission raises issues regardless of copyright. For a business, the practical posture is caution — understand the provenance and licensing of the tools you use, be careful with anything resembling real people or protected work, and get specifics from counsel before commercial use. This is law catching up to capability, and it's moving.
Separating transformation from hype
Stepping back across the whole course: generative AI beyond text is genuinely transformative for creative work, accessibility, and productivity — and genuinely over-hyped in the timelines and the "it can do anything" framing. The grounded view, modality by modality: image and speech are mature and broadly useful; music and video are impressive but carry legal and reliability caveats; 3D and long-form video are early. Match your investment to where each modality actually is, demand evidence over demos, and you'll capture the real value while sidestepping the expensive disappointments.
In one line each
- Generative media ends "seeing is believing" — real harms today (fraud, non-consensual imagery, disinformation) plus the liar's dividend.
- Detecting fakes is a losing arms race; don't base trust decisions on a detector.
- Provenance (sign and watermark at creation, e.g. C2PA, SynthID) is the more durable defence — but it proves origin, not truth, and only where adopted.
- Copyright, consent, and likeness are genuinely unsettled; know your tools' provenance and terms, and get counsel before commercial use.