Home › Hard problems › Edge AI and serving cost

Edge AI and serving cost: AI on small devices and cheaper answers

Always-on sensing on tiny chips (TinyML)

Solo: strongOne person can run a real test of this in about 8 weeks.

A useful modelas it is usuallytrainedthousands oftimes smallerKBa ₹500 board: kilobytes of memory+a coin batterya machine humminglistening all day for a fault
Always-on sensing means running a useful model on a chip with kilobytes of memory and a coin battery, so the model must be thousands of times smaller.

Why it is still open

Microcontrollers have kilobytes of memory and run on coin batteries. Useful models must be thousands of times smaller.

An experiment one person can run in 8 weeks

Train a sound or vibration detector (machine fault, cough) that fits under 256 KB and runs on a ₹500 board. Measure accuracy and battery life.

Verdict: Solo: strong

Cheap hardware and a demo anyone understands.

The solo test behind the verdict

  1. Can it be measured? There is a number that shows you did better, such as an error rate, a catch rate or a cost per answer.
  2. Is it cheap to try? It runs on a laptop, free open models and open data, or under about ₹50,000 of rented GPU time.
  3. Is there a public yardstick? A benchmark or dataset others use, so your result can be compared and believed.
  4. Would a small win get noticed? A lab, a funder or a customer would care about a 10% improvement, not only a full solution.

Four yeses means Solo: strong. Two or three means Solo: possible. Fewer means Follow, don't build.

PreviousThe memory cost of long conversations (the KV cache)NextLearning on the device while keeping data private

All 79 hard problems · Working on this, or know of a result we missed? basant@cleandataengineering.ai