Home › Hard problems › Inside the model

Inside the model: the core of LLMs

Understanding long video

Solo: possibleOne person can test a narrow slice of it.

dooropensphonecallcarleaves0 min60 minOne short clip"a man opens a door"?"Which came first:the call or the car leaving?"across the hour, the order of events gets lostcar, door, call
Models describe a short clip well, but lose track of who did what, and in which order, across an hour of video.

Why it is still open

Models describe a short clip well but lose track of what happened across an hour of video: who did what, in which order.

An experiment one person can run in 8 weeks

Ask 200 order-of-events questions about 30-minute public videos. Compare frame sampling against captions-then-text.

Verdict: Solo: possible

Video processing is costly, so keep the set small.

The solo test behind the verdict

  1. Can it be measured? There is a number that shows you did better, such as an error rate, a catch rate or a cost per answer.
  2. Is it cheap to try? It runs on a laptop, free open models and open data, or under about ₹50,000 of rented GPU time.
  3. Is there a public yardstick? A benchmark or dataset others use, so your result can be compared and believed.
  4. Would a small win get noticed? A lab, a funder or a customer would care about a 10% improvement, not only a full solution.

Four yeses means Solo: strong. Two or three means Solo: possible. Fewer means Follow, don't build.

PreviousPersonal memory across months of conversationsNextRunning out of human text; synthetic data collapse

All 79 hard problems · Working on this, or know of a result we missed? basant@cleandataengineering.ai