Home › Hard problems › Inside the model

Inside the model: the core of LLMs

World models and cause and effect

Solo: possibleOne person can test a narrow slice of it.

SEEN IN TRAINING, A THOUSAND TIMESpushpush the first one: all fallNEVER SEEN: TAKE ONE OUT FIRSTreally: two fall, three stay up"They all fall!"
Models learn patterns, not how the world works, so they fail on "what would happen if" questions they never saw in training.

Why it is still open

Models learn patterns, not how the world works. They fail on "what would happen if" questions outside their training data.

An experiment one person can run in 8 weeks

Build a small simulated world (a supply chain, a game) and test whether models predict what happens after an intervention they haven't seen.

Verdict: Solo: possible

The simulator becomes a reusable benchmark; links to AgentTwin.

The solo test behind the verdict

  1. Can it be measured? There is a number that shows you did better, such as an error rate, a catch rate or a cost per answer.
  2. Is it cheap to try? It runs on a laptop, free open models and open data, or under about ₹50,000 of rented GPU time.
  3. Is there a public yardstick? A benchmark or dataset others use, so your result can be compared and believed.
  4. Would a small win get noticed? A lab, a funder or a customer would care about a 10% improvement, not only a full solution.

Four yeses means Solo: strong. Two or three means Solo: possible. Fewer means Follow, don't build.

PreviousEvaluation itself is brokenNextNew architectures beyond the transformer

All 79 hard problems · Working on this, or know of a result we missed? basant@cleandataengineering.ai