Home › Hard problems › Society and trust

Society and trust

Spotting deepfakes

Solo: strongOne person can run a real test of this in about 8 weeks.

old generator 1old generator 2newest generatordetectortrained on old fakescaught: it learned their tell-tale markspasses as real: new marks, unseen
A deepfake detector learns the marks of the generators it has seen, so every new generator slips past it.

Why it is still open

Detectors break with every new generator.

An experiment one person can run in 8 weeks

Train a detector on images from older generators and measure how it fails on the newest ones. The size of that drop is the finding.

Verdict: Solo: strong

A public benchmark and a clear number.

The solo test behind the verdict

  1. Can it be measured? There is a number that shows you did better, such as an error rate, a catch rate or a cost per answer.
  2. Is it cheap to try? It runs on a laptop, free open models and open data, or under about ₹50,000 of rented GPU time.
  3. Is there a public yardstick? A benchmark or dataset others use, so your result can be compared and believed.
  4. Would a small win get noticed? A lab, a funder or a customer would care about a 10% improvement, not only a full solution.

Four yeses means Solo: strong. Two or three means Solo: possible. Fewer means Follow, don't build.

NextProving where content came from

All 79 hard problems · Working on this, or know of a result we missed? basant@cleandataengineering.ai