LLM Cost Explainer

Explains why LLM spend changed and which mistakes cost money, from a provider usage export alone.

Active · join meCost

6 of 11 tasks done

1 · Purpose

LLM Cost Explainer reads the usage export from an LLM provider and explains in plain words why spend went up or down. It splits the change into volume, model mix, request size, caching and price, then drills down to the model, key or project behind it. Rules then name common costly mistakes, such as caching turned off or bulk work on the real-time API, with an estimated saving. It is for teams building on LLMs who see a bill change and cannot tell why; it needs no SDK and never reads prompt text.

For: Engineering and finance teams running applications on hosted LLM APIs.

2 · Business benefits

Find wasted spendRules point to fixable mistakes and estimate what each fix would save.
Faster answers on bill changesA team gets the cause of a cost jump in minutes instead of a day of spreadsheet work.
No privacy riskOnly usage numbers are read; prompt text is never needed.
Numbers you can checkNo model computes a number; every figure comes from tested arithmetic.

3 · Status and how to join

Active · join me. Under way: pick a task below, or tell me what you'd like to learn and I'll suggest one. Lead: Basant Choudhary.

Good first tasks this week:

  • Add a new synthetic scenario with a known cause and a test for it.
  • Implement one more rule from the catalogue that needs only export data.
  • Write the fetch command for one provider.

The code is not public yet; joiners get access.

4 · Requirements

What it must do

How well it must do it

5 · Architecture, very high level

ImporterTurns provider usage pages intoone usage table.Price bookPrices tokens from a pinnedpublic price list.Change splitterCompares with a baseline andsplits the change by cause.Drill-downFinds which model, key orproject explains the change.Trend checkFlags unusual days against thesame weekday.Rule engineDetects costly patterns andestimates savings.ExplainerWrites the text and HTMLreport.usage bucketspriced usagechange by causwhy and saving
Components and what each is responsible for, with no tool names: the tools come in the next section.

Also: Price book → Trend check: daily cost · Price book → Rule engine: priced usage · Drill-down → Explainer: where

6 · Tech stack, per platform

ComponentOpen stackAWSAzure
ImporterPython 3.12 ✓AWS Lambda behind an upload pageAzure Functions behind an upload page
Price bookLiteLLM price map + DuckDB ✓Amazon S3 + DuckDBAzure Blob Storage + DuckDB
Change splitterPython, LMDI decomposition ✓——
Drill-downPython, Adtributor method ✓——
Trend checkPython, robust z-score ✓——
Rule engineYAML rule specs + SQL on DuckDB ✓——
ExplainerText templates + HTML ✓Static report on Amazon S3Static report on Azure Storage

Bold with ✓ = built and running today. Everything else is the planned or possible binding: the components stay the same, only the tools change.

7 · What needs to be done

Design 2 of 2 done

TaskStatusSizeSkill
Data model, cost equation and rule cataloguedoneLFinOps
Ingestion, normalisation, analysis and delivery designdoneLarchitecture

v0: export upload 4 of 4 done

TaskStatusSizeSkill
Importers for two providersdoneMPython
Pricing, five-way split, drill-down and trend checkdoneManalytics
Six rules with savings estimatesdoneMSQL
Text and HTML report, synthetic scenarios and testsdoneMPython

Real data 0 of 3 done

TaskStatusSizeSkill
Run on a real export and fix the field mappingtodoSdata engineering
Fetch command to pull usage pages with an admin keytodoSPython / APIs
Reconcile to the providers' cost endpoints: billed versus listtodoMFinOps

More rules and delivery 0 of 2 done

TaskStatusSizeSkill
Add rules from the remaining catalogue that the export can supporttodoMSQL
Upload page around the analysistodoMweb

Size: S = an evening or two, M = about a week part-time, L = a few weeks.

8 · Roles

RoleHow manyHours / weekWhat they do
Data / FinOps engineer13-4Adds rules and reconciles list price to the bill.
Backend engineer12-3Builds the fetch command and the upload page.
Tester with real data11Runs it on a real export and reports mapping gaps.

9 · What you'll learn

Level-up questions

10 · Links