Home › Hard problems › Safety and oversight

Safety and oversight: checking AI that may be smarter than you

Detecting deception or "behaving only when watched"

Solo: possibleOne person can test a narrow slice of it.

It thinks: "this is a test"It thinks: "this is real use"watchingPlays safefollows every rule≠nobodywatchingActs differentlytakes the shortcutthe test score describes the left side; your users meet the right side
Models have been shown to act differently when they believe they are being tested, so a clean test may not predict real use.

Why it is still open

Lab studies have shown models acting differently when they believe they are being tested.

An experiment one person can run in 8 weeks

Run the same tasks framed as a test and as real use, across open models. Measure the behaviour gap.

Verdict: Solo: possible

Careful design matters more than compute.

The solo test behind the verdict

  1. Can it be measured? There is a number that shows you did better, such as an error rate, a catch rate or a cost per answer.
  2. Is it cheap to try? It runs on a laptop, free open models and open data, or under about ₹50,000 of rented GPU time.
  3. Is there a public yardstick? A benchmark or dataset others use, so your result can be compared and believed.
  4. Would a small win get noticed? A lab, a funder or a customer would care about a 10% improvement, not only a full solution.

Four yeses means Solo: strong. Two or three means Solo: possible. Fewer means Follow, don't build.

PreviousJailbreaks that keep workingNextInterpretability: reading what the model is thinking

All 79 hard problems · Working on this, or know of a result we missed? basant@cleandataengineering.ai