LLM Cost Explainer
Explains why LLM spend changed and which mistakes cost money, from a provider usage export alone.
1 · Purpose
LLM Cost Explainer reads the usage export from an LLM provider and explains in plain words why spend went up or down. It splits the change into volume, model mix, request size, caching and price, then drills down to the model, key or project behind it. Rules then name common costly mistakes, such as caching turned off or bulk work on the real-time API, with an estimated saving. It is for teams building on LLMs who see a bill change and cannot tell why; it needs no SDK and never reads prompt text.
2 · Business benefits
3 · Status and how to join
Good first tasks this week:
- Add a new synthetic scenario with a known cause and a test for it.
- Implement one more rule from the catalogue that needs only export data.
- Write the fetch command for one provider.
4 · Requirements
What it must do
- Import usage pages from the OpenAI and Anthropic admin usage APIs into one usage table.
- Price tokens from a pinned, dated public price map, never from the source's own cost field.
- Compare a day against a baseline, such as the median of the last four same weekdays.
- Split the change five ways: volume, mix, size, cache and tier, and price.
- Drill down to the model, key, project or tier that explains the change.
- Flag unusual days and apply six cost rules with a savings estimate each.
- Write the explanation as text and as an HTML report.
How well it must do it
- No prompt text is ever read or stored.
- Fields that cannot be mapped are listed under data quality, never dropped silently.
- Each savings estimate prints the assumption behind it.
- Runs locally with one command; includes synthetic scenarios with known causes for testing.
5 · Architecture, very high level
6 · Tech stack, per platform
| Component | Open stack | AWS | Azure |
|---|---|---|---|
| Importer | Python 3.12 ✓ | AWS Lambda behind an upload page | Azure Functions behind an upload page |
| Price book | LiteLLM price map + DuckDB ✓ | Amazon S3 + DuckDB | Azure Blob Storage + DuckDB |
| Change splitter | Python, LMDI decomposition ✓ | — | — |
| Drill-down | Python, Adtributor method ✓ | — | — |
| Trend check | Python, robust z-score ✓ | — | — |
| Rule engine | YAML rule specs + SQL on DuckDB ✓ | — | — |
| Explainer | Text templates + HTML ✓ | Static report on Amazon S3 | Static report on Azure Storage |
7 · What needs to be done
Design 2 of 2 done
| Task | Status | Size | Skill |
|---|---|---|---|
| Data model, cost equation and rule catalogue | done | L | FinOps |
| Ingestion, normalisation, analysis and delivery design | done | L | architecture |
v0: export upload 4 of 4 done
| Task | Status | Size | Skill |
|---|---|---|---|
| Importers for two providers | done | M | Python |
| Pricing, five-way split, drill-down and trend check | done | M | analytics |
| Six rules with savings estimates | done | M | SQL |
| Text and HTML report, synthetic scenarios and tests | done | M | Python |
Real data 0 of 3 done
| Task | Status | Size | Skill |
|---|---|---|---|
| Run on a real export and fix the field mapping | todo | S | data engineering |
| Fetch command to pull usage pages with an admin key | todo | S | Python / APIs |
| Reconcile to the providers' cost endpoints: billed versus list | todo | M | FinOps |
More rules and delivery 0 of 2 done
| Task | Status | Size | Skill |
|---|---|---|---|
| Add rules from the remaining catalogue that the export can support | todo | M | SQL |
| Upload page around the analysis | todo | M | web |
8 · Roles
| Role | How many | Hours / week | What they do |
|---|---|---|---|
| Data / FinOps engineer | 1 | 3-4 | Adds rules and reconciles list price to the bill. |
| Backend engineer | 1 | 2-3 | Builds the fetch command and the upload page. |
| Tester with real data | 1 | 1 | Runs it on a real export and reports mapping gaps. |
9 · What you'll learn
- How to split a cost change into volume, mix, size, cache and price.
- How to find which slice of the data explains a change.
- What drives LLM spend: model choice, caching, batch versus real-time, service tier.
- Why list price and the bill differ.
Level-up questions
- How close is list-price cost to the real bill for a typical team?
- Which costly patterns can be seen from an export alone, and which need live traces?
- How should overlapping findings be counted so savings are not double-counted?
10 · Links
- Links come once the work is public.