Home › Hard problems › Edge AI and serving cost

Edge AI and serving cost: AI on small devices and cheaper answers

Learning on the device while keeping data private

Solo: possibleOne person can test a narrow slice of it.

The shared modelon a serverthe data stays on each phone;only model updates travellots of datavery little datawent offlinean attackera poisoned updatedifferent habits
Training across many phones keeps the data private, but the phones are very different, they drop out, and some send poisoned updates.

Why it is still open

Federated learning (training across many phones without collecting their data) struggles with very different users, unreliable devices and attacks.

An experiment one person can run in 8 weeks

Simulate 1,000 phones with uneven data and compare 3 methods on accuracy and privacy leakage.

Verdict: Solo: possible

Simulation is cheap; real fleets need a partner.

The solo test behind the verdict

  1. Can it be measured? There is a number that shows you did better, such as an error rate, a catch rate or a cost per answer.
  2. Is it cheap to try? It runs on a laptop, free open models and open data, or under about ₹50,000 of rented GPU time.
  3. Is there a public yardstick? A benchmark or dataset others use, so your result can be compared and believed.
  4. Would a small win get noticed? A lab, a funder or a customer would care about a 10% improvement, not only a full solution.

Four yeses means Solo: strong. Two or three means Solo: possible. Fewer means Follow, don't build.

PreviousAlways-on sensing on tiny chips (TinyML)NextEnergy per answer

All 79 hard problems · Working on this, or know of a result we missed? basant@cleandataengineering.ai