Home › Hard problems › Chips, compute and energy

Chips, compute and energy

Connecting thousands of chips

Follow, don't buildNeeds labs, hardware or big teams. Learn it and watch it.

one big model, spread over thousands of chipsrack Arack B: chips idle, waitingdata piles up between chipstimepoweroptical links and new network designs: still being proven
Big models span many chips, and every move of data between them costs time and power.

Why it is still open

Big models span many chips, and moving data between them costs time and power. Optical links and new network designs are still being proven.

An experiment one person can run in 8 weeks

No solo build. Learn how NVLink, InfiniBand and optical links compare.

Verdict: Follow, don't build

Needs hardware labs.

The solo test behind the verdict

  1. Can it be measured? There is a number that shows you did better, such as an error rate, a catch rate or a cost per answer.
  2. Is it cheap to try? It runs on a laptop, free open models and open data, or under about ₹50,000 of rented GPU time.
  3. Is there a public yardstick? A benchmark or dataset others use, so your result can be compared and believed.
  4. Would a small win get noticed? A lab, a funder or a customer would care about a 10% improvement, not only a full solution.

Four yeses means Solo: strong. Two or three means Solo: possible. Fewer means Follow, don't build.

PreviousThe memory wallNextPower and cooling for AI data centres

All 79 hard problems · Working on this, or know of a result we missed? basant@cleandataengineering.ai