Home › Paths › LLM Engineering

Learning path

LLM Engineering

You work on the models themselves: data, tuning, serving and cost per token.

Why this path

AI engineers use models; LLM engineers shape them. That means knowing how a model is trained, how to adapt one to a task, how to serve it fast and cheap, and when a small model beats a big one.

How to use this page: 1. learn the key concepts, which are the same everywhere; 2. pick your platform; 3. work through Crawl, Walk and Run.

Read alongside: The AI Universe · Agent Harness, drawn simply · Certification prep on this site.

Step 1 · the same everywhere

Key concepts

Learn these ideas first. They are the same on every cloud and with every tool; only the product names change. Each one is a short page: what it is, why it matters, an example and the common mistake.

Step 2

Where do you work?

Pick your platform and you get one page with that platform's products in every step.

Beginner · 6 weeks

Crawl: You can build one thing end to end, with help.

Understand how a language model works by building a tiny one, then use real ones well.

Key ideas at this level, the same on every platform:

Open this level on: AWS · Azure · Google Cloud · Databricks · Open source

Learn: the concept, and the product on each platform

LearnAWSAzureGoogle CloudDatabricksOpen source
How a model workstokens, embeddings, attention, next-token predictionPythonPythonPythonPythonPython
Tiny model by handtrain a very small model on your laptopSageMaker training jobsAzure ML jobsVertex AI trainingDatabricks ML runtimescikit-learn, PyTorch
Using hosted modelsprompts, context windows, cost and speedAmazon Bedrock
Bedrock Converse API
Azure OpenAI in Microsoft Foundry
Foundry structured outputs
Gemini on Vertex AI
Gemini controlled generation
Foundation Model APIs
ai_query + structured output
Ollama, or a free hosted open model
Pydantic + JSON mode
Embeddingsturn text into vectors; search by meaningOpenSearch Serverless / Aurora pgvectorAzure AI SearchVertex AI Vector Search / AlloyDBVector Searchpgvector or Qdrant

Build on GitHub

Under way · join

LearnLLMStack

A tiny version of every layer of an LLM product, from your own small model up to the app, in one stack.

Proposed · needs a lead

Ask your documents, with citations

Search a folder of PDFs and answer with the exact passage cited, plus a 50-question test set you run on every change.

You are ready to move on when

Intermediate · 3 months

Walk: You can build it so it keeps working, without help.

Adapt and serve models: fine-tune, host open weights and measure quality and cost.

Key ideas at this level, the same on every platform:

Open this level on: AWS · Azure · Google Cloud · Databricks · Open source

Learn: the concept, and the product on each platform

LearnAWSAzureGoogle CloudDatabricksOpen source
Fine-tuningwhen to tune and when to prompt; LoRA; good examplesBedrock custom models / SageMakerFoundry fine-tuningVertex AI tuningMosaic AI fine-tuningHugging Face TRL or Unsloth
Serving open modelsbatching, quantisation, throughputSageMaker endpoints / EKS + vLLMFoundry managed compute / AKS + vLLMVertex AI endpoints / GKE + vLLMProvisioned throughput servingvLLM
Evaluationtask metrics, judge models, human reviewBedrock evaluationsFoundry evaluationsVertex AI Gen AI evaluationMLflow evaluationEval Harness Framework or promptfoo
Cost per tokenmeasure it, then cut it with caching and routingBedrock cross-region inference + API Gateway
Cost Explorer + CUR
API Management AI gateway
Cost Management
Apigee
Billing export to BigQuery
Mosaic AI Gateway
Billing system tables
LiteLLM
Query plans and run times

Build on GitHub

Proposed · needs a lead

Fine-tune a small model

Teach a small open model one narrow task from a big model's examples, then compare quality, speed and cost honestly.

Under way · join

LLM cost explainer

Turn a provider's usage export into a plain report: where the money went and what to change.

Just starting · join

MyClaudeCode

A small coding agent built from scratch, to learn how context is budgeted and how tools are kept safe.

You are ready to move on when

Advanced · 6 months +

Run: You can design it, prove it and lead others through it.

Build the data and training pipeline for a model, and publish it.

Key ideas at this level, the same on every platform:

Open this level on: AWS · Azure · Google Cloud · Databricks · Open source

Learn: the concept, and the product on each platform

LearnAWSAzureGoogle CloudDatabricksOpen source
Training data at scalecollect, clean, dedupe, filter, licenseEMR / Glue on S3
Amazon S3
Fabric Spark on OneLake
ADLS Gen2 / OneLake
Dataflow / Dataproc on GCS
Cloud Storage
Spark on Delta
Unity Catalog volumes on cloud storage
datatrove or Spark
Local disk or MinIO
Pre-training and post-trainingcontinued pre-training, instruction tuning, preference tuningBedrock custom models / SageMaker
SageMaker endpoints / EKS + vLLM
Foundry fine-tuning
Foundry managed compute / AKS + vLLM
Vertex AI tuning
Vertex AI endpoints / GKE + vLLM
Mosaic AI fine-tuning
Provisioned throughput serving
Hugging Face TRL or Unsloth
vLLM
Benchmarkshonest evaluation in a new language or domainBedrock evaluationsFoundry evaluationsVertex AI Gen AI evaluationMLflow evaluationEval Harness Framework or promptfoo
Safetyred-teaming and guardrails for your modelBedrock GuardrailsAzure AI Content SafetyModel ArmorAI Gateway guardrailsrule checks in your own code

Build on GitHub

Needs a lead

Hinglish pre-training data

Build a clean, licensed, de-duplicated Hinglish text corpus and a small model trained on it.

Needs a lead

Robotics data benchmark

A benchmark for the data pipelines that feed robot learning: formats, quality and cost.

You are ready to move on when