Home › Paths › LLM Engineering › Open source
Learning path · Open source
LLM Engineering on Open source
You work on the models themselves: data, tuning, serving and cost per token.
Free, like everything here: no sign-up, no paywall. Only the cloud you practise on may charge; the open-source path costs nothing.
AI engineers use models; LLM engineers shape them. That means knowing how a model is trained, how to adapt one to a task, how to serve it fast and cheap, and when a small model beats a big one.
Every topic below is a concept (the job) with the Open source product that does it. Learn the concept once; on another platform only the product name changes. Read alongside: The AI Universe · Agent Harness, drawn simply · Certification prep on this site.
Beginner · 6 weeks
Crawl: You can build one thing end to end, with help.
Understand how a language model works by building a tiny one, then use real ones well.
Key ideas at this level, the same on every platform:
Learn
| Learn | What it means | On Open source you use |
|---|---|---|
| How a model works | tokens, embeddings, attention, next-token prediction | Pythona language, properly |
| Tiny model by hand | train a very small model on your laptop | scikit-learn, PyTorchtraining |
| Using hosted models | prompts, context windows, cost and speed | Ollama, or a free hosted open modelmodel api Pydantic + JSON modeprompts and structured output |
| Embeddings | turn text into vectors; search by meaning | pgvector or Qdrantvector search |
Build on GitHub
LearnLLMStack
A tiny version of every layer of an LLM product, from your own small model up to the app, in one stack.
Built on Open source already.
Ask your documents, with citations
Search a folder of PDFs and answer with the exact passage cited, plus a 50-question test set you run on every change.
You are ready to move on when
- You can explain attention to a friend with a drawing.
- Your search finds the right passage for 8 of 10 test questions.
- You know the cost per 1,000 answers.
Intermediate · 3 months
Walk: You can build it so it keeps working, without help.
Adapt and serve models: fine-tune, host open weights and measure quality and cost.
Key ideas at this level, the same on every platform:
Learn
| Learn | What it means | On Open source you use |
|---|---|---|
| Fine-tuning | when to tune and when to prompt; LoRA; good examples | Hugging Face TRL or Unslothfine-tuning |
| Serving open models | batching, quantisation, throughput | vLLMmodel hosting on gpus |
| Evaluation | task metrics, judge models, human review | Eval Harness Framework or promptfooevaluation |
| Cost per token | measure it, then cut it with caching and routing | LiteLLMmodel gateway Query plans and run timescost |
Build on GitHub
Fine-tune a small model
Teach a small open model one narrow task from a big model's examples, then compare quality, speed and cost honestly.
LLM cost explainer
Turn a provider's usage export into a plain report: where the money went and what to change.
Built on Open source already.
MyClaudeCode
A small coding agent built from scratch, to learn how context is budgeted and how tools are kept safe.
Built on Open source already.
You are ready to move on when
- Your small tuned model matches the big one on your task at a fraction of the cost.
- You can serve it and state the tokens per second.
- Your evaluation would catch a worse model.
Advanced · 6 months +
Run: You can design it, prove it and lead others through it.
Build the data and training pipeline for a model, and publish it.
Key ideas at this level, the same on every platform:
Learn
| Learn | What it means | On Open source you use |
|---|---|---|
| Training data at scale | collect, clean, dedupe, filter, license | datatrove or Sparktraining data at scale Local disk or MinIOobject storage |
| Pre-training and post-training | continued pre-training, instruction tuning, preference tuning | Hugging Face TRL or Unslothfine-tuning vLLMmodel hosting on gpus |
| Benchmarks | honest evaluation in a new language or domain | Eval Harness Framework or promptfooevaluation |
| Safety | red-teaming and guardrails for your model | rule checks in your own codeguardrails |
Build on GitHub
Hinglish pre-training data
Build a clean, licensed, de-duplicated Hinglish text corpus and a small model trained on it.
Robotics data benchmark
A benchmark for the data pipelines that feed robot learning: formats, quality and cost.
You are ready to move on when
- Your dataset has a data card, a licence and a de-duplication report.
- You have trained and evaluated a model on it.
- Others can reproduce your result.
Optional
Certifications on Open source
Projects prove more than certificates, but a certificate can open a door. These fit this path. Exam names and codes change, so check the current one before you book.
- Claude Certified Architect – Foundations
Free prep we have written: Certifications.
Learn together
Do it with others
Every project above has a page, a list of open work and a "good first issue" label on GitHub. Pick one at your level, say hello by email, and build it in the open. If a project says needs a lead, that can be you.