Home › Paths › LLM Engineering
Learning path
LLM Engineering
You work on the models themselves: data, tuning, serving and cost per token.
Why this path
AI engineers use models; LLM engineers shape them. That means knowing how a model is trained, how to adapt one to a task, how to serve it fast and cheap, and when a small model beats a big one.
How to use this page: 1. learn the key concepts, which are the same everywhere; 2. pick your platform; 3. work through Crawl, Walk and Run.
Read alongside: The AI Universe · Agent Harness, drawn simply · Certification prep on this site.
Step 1 · the same everywhere
Key concepts
Learn these ideas first. They are the same on every cloud and with every tool; only the product names change. Each one is a short page: what it is, why it matters, an example and the common mistake.
How models work
Adapting models
Serving models
Quality and data
Step 2
Where do you work?
Pick your platform and you get one page with that platform's products in every step.
Beginner · 6 weeks
Crawl: You can build one thing end to end, with help.
Understand how a language model works by building a tiny one, then use real ones well.
Key ideas at this level, the same on every platform:
Open this level on: AWS · Azure · Google Cloud · Databricks · Open source
Learn: the concept, and the product on each platform
| Learn | AWS | Azure | Google Cloud | Databricks | Open source |
|---|---|---|---|---|---|
| How a model workstokens, embeddings, attention, next-token prediction | Python | Python | Python | Python | Python |
| Tiny model by handtrain a very small model on your laptop | SageMaker training jobs | Azure ML jobs | Vertex AI training | Databricks ML runtime | scikit-learn, PyTorch |
| Using hosted modelsprompts, context windows, cost and speed | Amazon Bedrock Bedrock Converse API | Azure OpenAI in Microsoft Foundry Foundry structured outputs | Gemini on Vertex AI Gemini controlled generation | Foundation Model APIs ai_query + structured output | Ollama, or a free hosted open model Pydantic + JSON mode |
| Embeddingsturn text into vectors; search by meaning | OpenSearch Serverless / Aurora pgvector | Azure AI Search | Vertex AI Vector Search / AlloyDB | Vector Search | pgvector or Qdrant |
Build on GitHub
LearnLLMStack
A tiny version of every layer of an LLM product, from your own small model up to the app, in one stack.
Ask your documents, with citations
Search a folder of PDFs and answer with the exact passage cited, plus a 50-question test set you run on every change.
You are ready to move on when
- You can explain attention to a friend with a drawing.
- Your search finds the right passage for 8 of 10 test questions.
- You know the cost per 1,000 answers.
Intermediate · 3 months
Walk: You can build it so it keeps working, without help.
Adapt and serve models: fine-tune, host open weights and measure quality and cost.
Key ideas at this level, the same on every platform:
Open this level on: AWS · Azure · Google Cloud · Databricks · Open source
Learn: the concept, and the product on each platform
| Learn | AWS | Azure | Google Cloud | Databricks | Open source |
|---|---|---|---|---|---|
| Fine-tuningwhen to tune and when to prompt; LoRA; good examples | Bedrock custom models / SageMaker | Foundry fine-tuning | Vertex AI tuning | Mosaic AI fine-tuning | Hugging Face TRL or Unsloth |
| Serving open modelsbatching, quantisation, throughput | SageMaker endpoints / EKS + vLLM | Foundry managed compute / AKS + vLLM | Vertex AI endpoints / GKE + vLLM | Provisioned throughput serving | vLLM |
| Evaluationtask metrics, judge models, human review | Bedrock evaluations | Foundry evaluations | Vertex AI Gen AI evaluation | MLflow evaluation | Eval Harness Framework or promptfoo |
| Cost per tokenmeasure it, then cut it with caching and routing | Bedrock cross-region inference + API Gateway Cost Explorer + CUR | API Management AI gateway Cost Management | Apigee Billing export to BigQuery | Mosaic AI Gateway Billing system tables | LiteLLM Query plans and run times |
Build on GitHub
Fine-tune a small model
Teach a small open model one narrow task from a big model's examples, then compare quality, speed and cost honestly.
LLM cost explainer
Turn a provider's usage export into a plain report: where the money went and what to change.
MyClaudeCode
A small coding agent built from scratch, to learn how context is budgeted and how tools are kept safe.
You are ready to move on when
- Your small tuned model matches the big one on your task at a fraction of the cost.
- You can serve it and state the tokens per second.
- Your evaluation would catch a worse model.
Advanced · 6 months +
Run: You can design it, prove it and lead others through it.
Build the data and training pipeline for a model, and publish it.
Key ideas at this level, the same on every platform:
Open this level on: AWS · Azure · Google Cloud · Databricks · Open source
Learn: the concept, and the product on each platform
| Learn | AWS | Azure | Google Cloud | Databricks | Open source |
|---|---|---|---|---|---|
| Training data at scalecollect, clean, dedupe, filter, license | EMR / Glue on S3 Amazon S3 | Fabric Spark on OneLake ADLS Gen2 / OneLake | Dataflow / Dataproc on GCS Cloud Storage | Spark on Delta Unity Catalog volumes on cloud storage | datatrove or Spark Local disk or MinIO |
| Pre-training and post-trainingcontinued pre-training, instruction tuning, preference tuning | Bedrock custom models / SageMaker SageMaker endpoints / EKS + vLLM | Foundry fine-tuning Foundry managed compute / AKS + vLLM | Vertex AI tuning Vertex AI endpoints / GKE + vLLM | Mosaic AI fine-tuning Provisioned throughput serving | Hugging Face TRL or Unsloth vLLM |
| Benchmarkshonest evaluation in a new language or domain | Bedrock evaluations | Foundry evaluations | Vertex AI Gen AI evaluation | MLflow evaluation | Eval Harness Framework or promptfoo |
| Safetyred-teaming and guardrails for your model | Bedrock Guardrails | Azure AI Content Safety | Model Armor | AI Gateway guardrails | rule checks in your own code |
Build on GitHub
Hinglish pre-training data
Build a clean, licensed, de-duplicated Hinglish text corpus and a small model trained on it.
Robotics data benchmark
A benchmark for the data pipelines that feed robot learning: formats, quality and cost.
You are ready to move on when
- Your dataset has a data card, a licence and a de-duplication report.
- You have trained and evaluated a model on it.
- Others can reproduce your result.