Home › Hard problems › Safety and oversight

Safety and oversight: checking AI that may be smarter than you

Models memorising private training data

Solo: strongOne person can run a real test of this in about 8 weeks.

training dataRavi: 98765 43210one note with a phonenumber in it (say)The modeltrained on it"Ravi's phone number is...""98765 43210"repeated word for wordThe known fix: train with privacy guaranteesit leaks less, but the answers lose a lot of quality
Models can repeat private training text, like phone numbers, keys or medical notes, word for word.

Why it is still open

Models can repeat phone numbers, keys or medical notes word for word. Training with privacy guarantees costs a lot of quality.

An experiment one person can run in 8 weeks

Plant 100 fake secrets in fine-tuning data and measure how many can be pulled out, with and without privacy training.

Verdict: Solo: strong

Cheap, clear numbers, and regulated buyers care.

The solo test behind the verdict

  1. Can it be measured? There is a number that shows you did better, such as an error rate, a catch rate or a cost per answer.
  2. Is it cheap to try? It runs on a laptop, free open models and open data, or under about ₹50,000 of rented GPU time.
  3. Is there a public yardstick? A benchmark or dataset others use, so your result can be compared and believed.
  4. Would a small win get noticed? A lab, a funder or a customer would care about a 10% improvement, not only a full solution.

Four yeses means Solo: strong. Two or three means Solo: possible. Fewer means Follow, don't build.

PreviousUnlearning: removing knowledge for good

All 79 hard problems · Working on this, or know of a result we missed? basant@cleandataengineering.ai