Home › Hard problems › Edge AI and serving cost

Edge AI and serving cost: AI on small devices and cheaper answers

Energy per answer

Solo: strongOne person can run a real test of this in about 8 weeks.

the grid? Whwatt-hours per answer:rarely measuredThe modelon a GPUor a laptopan answerbatch sizebitscache reusesettings thatmove the meterAI demandgrid supply
AI's electricity demand is growing faster than grids, yet few people measure what one answer costs in energy, end to end.

Why it is still open

AI's electricity demand is growing faster than grids, and few people measure the cost of a token end to end.

An experiment one person can run in 8 weeks

Measure watt-hours per answer for one model across settings (batch size, quantization, cache reuse) on one GPU and one laptop. Publish a cost and energy table.

Verdict: Solo: strong

A clear number that every buyer of AI cares about.

The solo test behind the verdict

  1. Can it be measured? There is a number that shows you did better, such as an error rate, a catch rate or a cost per answer.
  2. Is it cheap to try? It runs on a laptop, free open models and open data, or under about ₹50,000 of rented GPU time.
  3. Is there a public yardstick? A benchmark or dataset others use, so your result can be compared and believed.
  4. Would a small win get noticed? A lab, a funder or a customer would care about a 10% improvement, not only a full solution.

Four yeses means Solo: strong. Two or three means Solo: possible. Fewer means Follow, don't build.

PreviousLearning on the device while keeping data privateNextTrusting a model running on someone else's device

All 79 hard problems · Working on this, or know of a result we missed? basant@cleandataengineering.ai