Home › Hard problems › Inside the model

Inside the model: the core of LLMs

Personal memory across months of conversations

Solo: strongOne person can run a real test of this in about 8 weeks.

JanFebMarAprMayJun"I live in Pune""I'm vegetarian""I moved to Delhi"THE ASSISTANT'S MEMORYlives in Punelives in Delhiboth kept: which one is true now?health notemay leak"A gym near home?""Here are gyms in Pune!"
Memories go stale, contradict each other and can leak private details, and nobody has settled what an assistant should remember, update or forget.

Why it is still open

What should an assistant remember, forget, update or never store? Memories go stale, contradict each other and leak private details.

An experiment one person can run in 8 weeks

Build a 6-month synthetic user history with facts that change (moved city, new job). Measure how often 3 memory designs answer with the old fact.

Verdict: Solo: strong

Easy to measure, and every assistant product needs it.

The solo test behind the verdict

  1. Can it be measured? There is a number that shows you did better, such as an error rate, a catch rate or a cost per answer.
  2. Is it cheap to try? It runs on a laptop, free open models and open data, or under about ₹50,000 of rented GPU time.
  3. Is there a public yardstick? A benchmark or dataset others use, so your result can be compared and believed.
  4. Would a small win get noticed? A lab, a funder or a customer would care about a 10% improvement, not only a full solution.

Four yeses means Solo: strong. Two or three means Solo: possible. Fewer means Follow, don't build.

PreviousUsing long context, not just fitting itNextUnderstanding long video

All 79 hard problems · Working on this, or know of a result we missed? basant@cleandataengineering.ai