Learning path
AI Engineering
You build products and agents where a model does part of the work, safely.
Why this path
Most of an AI system is ordinary software around a small model call: the tools, the checks, the state, the evaluation and the bill. AI engineers build that harness so the product holds up after the demo.
How to use this page: 1. learn the key concepts, which are the same everywhere; 2. pick your platform; 3. work through Crawl, Walk and Run.
Read alongside: The AI Universe · Agent Harness, drawn simply · The Agent Harness Series · Eval Harness, drawn simply · Vertical agents, by industry · Certification prep on this site.
Step 1 · the same everywhere
Key concepts
Learn these ideas first. They are the same on every cloud and with every tool; only the product names change. Each one is a short page: what it is, why it matters, an example and the common mistake.
Working with models
Tools and agents
Knowledge
Safety and proof
Engineering basics
Step 2
Where do you work?
Pick your platform and you get one page with that platform's products in every step.
Beginner · 6 weeks
Crawl: You can build one thing end to end, with help.
Call a model, get checked structured output, and give it a few safe tools.
Key ideas at this level, the same on every platform:
Open this level on: AWS · Azure · Google Cloud · Databricks · Open source
Learn: the concept, and the product on each platform
| Learn | AWS | Azure | Google Cloud | Databricks | Open source |
|---|---|---|---|---|---|
| Python and APIsHTTP, JSON, async, tests; read an API's docs | Python Lambda / ECS Fargate | Python Container Apps / Functions | Python Cloud Run | Python Databricks Apps | Python Docker |
| Calling a modelmessages, tokens, temperature, cost per call | Amazon Bedrock | Azure OpenAI in Microsoft Foundry | Gemini on Vertex AI | Foundation Model APIs | Ollama, or a free hosted open model |
| Structured outputask for JSON, validate it, retry on failure | Bedrock Converse API | Foundry structured outputs | Gemini controlled generation | ai_query + structured output | Pydantic + JSON mode |
| Toolslet the model ask your code to act; check before acting | Bedrock tool use | Foundry function calling | Gemini function calling | Unity Catalog functions as tools | plain Python loop, no framework |
| Deterministic firstif code can do it, code does it; the model handles the rest | Python | Python | Python | Python | Python |
Build on GitHub
Agent harness from scratch
Build the loop, tools, context and checks around a model in plain Python, one small step at a time.
Your first agent, three tools
A support assistant that can look up an order, check a policy and draft a reply, where code checks every tool call first.
You are ready to move on when
- You can say what each call cost.
- A wrong tool call is refused by code, not by hope.
- You have ten written test cases and they pass.
Intermediate · 3 months
Walk: You can build it so it keeps working, without help.
Build the harness: retrieval, state, approvals, tracing and evaluation on every change.
Key ideas at this level, the same on every platform:
Open this level on: AWS · Azure · Google Cloud · Databricks · Open source
Learn: the concept, and the product on each platform
| Learn | AWS | Azure | Google Cloud | Databricks | Open source |
|---|---|---|---|---|---|
| Retrievalsearch by meaning; chunking; citations | OpenSearch Serverless / Aurora pgvector | Azure AI Search | Vertex AI Vector Search / AlloyDB | Vector Search | pgvector or Qdrant |
| Agent loop and statesteps, budgets, checkpoints, waiting for a human | Bedrock AgentCore | Foundry Agent Service / Durable Functions | Vertex AI Agent Engine | Mosaic AI Agent Framework | DBOS or Temporal + your loop |
| Checks and guardrailsbefore and after every model and tool call | Bedrock Guardrails | Azure AI Content Safety | Model Armor | AI Gateway guardrails | rule checks in your own code |
| Tracingevery call, token and cent visible | CloudWatch + X-Ray (OpenTelemetry) | App Insights (OpenTelemetry) | Cloud Trace (OpenTelemetry) | MLflow tracing | Phoenix or Langfuse |
| Evaluationgolden cases, rules, a gate in CI | Bedrock evaluations GitHub Actions + CodeDeploy | Foundry evaluations GitHub Actions or Azure Pipelines | Vertex AI Gen AI evaluation GitHub Actions + Cloud Build | MLflow evaluation GitHub Actions + bundles | Eval Harness Framework or promptfoo GitHub Actions |
Build on GitHub
Claims assistant on Azure
A motor-insurance claims agent running for real: tools, checks, approvals, durable waits, evaluation and cost.
LearnLangGraph
A hands-on course in graph-shaped agents: state, branches, human approval and resuming after a crash.
Reference agent
A customer-support agent with no framework: one module per harness layer, the architecture enforced by tests.
You are ready to move on when
- A crash mid-conversation resumes without repeating a payment.
- A release is blocked when evaluation scores drop.
- You can open one trace and explain the whole turn.
Advanced · 6 months +
Run: You can design it, prove it and lead others through it.
Prove an agent before it meets real people, and run many of them at a known cost.
Key ideas at this level, the same on every platform:
Open this level on: AWS · Azure · Google Cloud · Databricks · Open source
Learn: the concept, and the product on each platform
| Learn | AWS | Azure | Google Cloud | Databricks | Open source |
|---|---|---|---|---|---|
| Simulated worldstest against simulated customers, systems and faults | Bedrock evaluations | Foundry evaluations | Vertex AI Gen AI evaluation | MLflow evaluation | Eval Harness Framework or promptfoo |
| Gateway and budgetsmany models behind one door; fallbacks; spend limits | Bedrock cross-region inference + API Gateway | API Management AI gateway | Apigee | Mosaic AI Gateway | LiteLLM |
| Assurancewhat must be true before it is trusted; evidence, not claims | Bedrock evaluations Bedrock Guardrails | Foundry evaluations Azure AI Content Safety | Vertex AI Gen AI evaluation Model Armor | MLflow evaluation AI Gateway guardrails | Eval Harness Framework or promptfoo rule checks in your own code |
| Portabilitysame agent, new cloud, by swapping adapters | Terraform or AWS CDK Bedrock AgentCore | Bicep or Terraform Foundry Agent Service / Durable Functions | Terraform Vertex AI Agent Engine | Declarative Automation Bundles (Asset Bundles) Mosaic AI Agent Framework | Docker Compose DBOS or Temporal + your loop |
| Researchlong-running agents, self-repair, drift | CloudWatch + X-Ray (OpenTelemetry) | App Insights (OpenTelemetry) | Cloud Trace (OpenTelemetry) | MLflow tracing | Phoenix or Langfuse |
Build on GitHub
Eval Harness Framework
Rebuild each conversation from the agent's own traces and check it against rules written in plain YAML.
AgentTwin
Simulate the agent's world (customers, systems, faults, time) so a real agent can be tested before it meets real people.
Port the agent to your cloud
Take the reference agent and bind it to AWS or Google Cloud: same code, new adapters, same tests passing.
AgentTwin Lab
Research: an agent runs for weeks in a simulated business while a fixer repairs it. Does the fixer cheat?
You are ready to move on when
- Your agent has passed a simulated week before going live.
- You can move it to another cloud and the same tests pass.
- You have published what you learned.