Do deepfake detectors work? What AI catches (and what slips through)

You got a suspicious video, dropped it into an online "deepfake detector", and the site declared it: 87% chance of being fake. Case closed? Not quite — and understanding why that percentage is worth less than it looks is the difference between using these tools productively and outsourcing your critical thinking to a system that errs with conviction.

The short answer

Deepfake and AI-text detectors work partially, under controlled conditions, against content similar to what they saw in training — and they degrade fast in the real world. They're one more signal in an investigation, never a verdict. Anyone promising "99% accuracy" is measuring in their own backyard.

The public numbers are eloquent. In the Deepfake Detection Challenge, a competition backed by Meta with thousands of entrants, the winning model got about 82% right on a known set of videos — and dropped to roughly 65% when tested on videos it had never seen. On the text side, OpenAI itself launched a classifier to tell human writing from AI writing and retired it months later, in 2023, admitting low accuracy. If the maker of the generator doesn't trust its own detector, be suspicious of anyone selling certainty.

Why detection is structurally hard

• It's a race where the defense runs behind: the detector learns the flaws of the current generation; the next generation is trained, sometimes literally, not to have those flaws. Every artifact that gives a deepfake away today — a shimmering edge, a strange blink — is an item on tomorrow's fix list.
• Compression erases the clues: the video that reaches your phone has been compressed two or three times. The subtle statistical patterns the detector looks for are exactly what compression destroys — what's left is a verdict about noise.
• False positives are expensive: in text, the documented damage is real. Stanford researchers showed that detectors flagged as "AI" more than half the essays written by students who are not native English speakers. Simple, regular writing — a student's, a non-native speaker's, anyone taught to write tidily — mimics the statistical regularity of the machine. An innocent student accused by software is the practical result.
• The percentage misleads: "87% chance of fake" sounds scientific, but it's the model's confidence within its own assumptions — not the real probability. On content outside its training, a high number may only mean the video is grainy, badly lit, or run through a beauty filter.

What works better than a detector

The productive question is rarely "is this media synthetic?" — it's "is this story true?". And that gets answered by verification, not by an oracle:

• Provenance instead of guesswork: the industry's serious bet is to sign content at the source — the C2PA standard (Content Credentials), already adopted by cameras and editing suites, and watermarks like Google's SynthID embedded at generation time. Instead of "does it look fake?", the question becomes "where did it come from and what was edited?". It doesn't solve everything (a missing signature proves nothing), but it flips the game in favor of verification.
• Context and source: who published it first? Does a trustworthy outlet confirm it? Does the supposed video of the public figure exist on their official channel? Ninety percent of viral deepfakes die at that two-minute check.
• The human tests that still hold: on a video call, asking for unexpected movement — a hand in front of the face, turning to profile — still breaks real-time face swaps, as we detail in our guide to spotting a deepfake. And the alternate channel — hang up and call back on a number you know — solves what no detector can.
• Process instead of eyeballs: companies burned by deepfake candidates in job interviews weren't saved by detection software, but by identity verification and least privilege. Against fraud, procedure beats perception.

Is it worth using an AI-text detector at school or at work?

As the sole evidence for accusing someone, no — the error rate condemns the innocent and clears anyone who edited half a dozen sentences. As triage to decide where to look closely, it can help, as long as whoever uses it knows the result is a statistical guess. The honest policy: assess what only a human can demonstrate (an oral defense, document version history, the process) instead of hunting down the origin of every paragraph.

The full picture

There's an irony in this race: AI is used on both sides of it. The same models that generate flawless phishing and the scams in our guide to AI fraud also feed the filters trying to block them. Automatic detection has a real role at scale — platforms filtering billions of uploads need it, flaws and all. What it can't be is your only individual line of defense.

The rule of thumb to take away: a detector is a signal, provenance is evidence, verification is the verdict. If a piece of media matters enough for you to act on it — to pay, to vote, to share, to accuse — it matters enough for two minutes of source checking. No site with a percentage bar replaces that.