Home › Hard problems › Safety and oversight

Safety and oversight: checking AI that may be smarter than you

Interpretability: reading what the model is thinking

Solo: possibleOne person can test a narrow slice of it.

The idea 'water'The idea 'boats'two ideas themodel clearly hasbillions of numbers, none of them labelledeach idea is smeared across many numbers, and one number can serve many ideaswhich ones mean 'water'?
A model is billions of unlabelled numbers, and each idea is smeared across many of them, so you can't just look one up.

Why it is still open

Billions of numbers with no labels, and concepts are smeared across many of them.

An experiment one person can run in 8 weeks

Use TransformerLens and published sparse autoencoders on a small open model. Find where one concept lives and test whether switching it off changes the behaviour.

Verdict: Solo: possible

Many solo researchers publish here, though it takes maths patience.

The solo test behind the verdict

  1. Can it be measured? There is a number that shows you did better, such as an error rate, a catch rate or a cost per answer.
  2. Is it cheap to try? It runs on a laptop, free open models and open data, or under about ₹50,000 of rented GPU time.
  3. Is there a public yardstick? A benchmark or dataset others use, so your result can be compared and believed.
  4. Would a small win get noticed? A lab, a funder or a customer would care about a 10% improvement, not only a full solution.

Four yeses means Solo: strong. Two or three means Solo: possible. Fewer means Follow, don't build.

PreviousDetecting deception or "behaving only when watched"NextUnlearning: removing knowledge for good

All 79 hard problems · Working on this, or know of a result we missed? basant@cleandataengineering.ai