Home › Paths › AI Engineering

Learning path

AI Engineering

You build products and agents where a model does part of the work, safely.

Why this path

Most of an AI system is ordinary software around a small model call: the tools, the checks, the state, the evaluation and the bill. AI engineers build that harness so the product holds up after the demo.

How to use this page: 1. learn the key concepts, which are the same everywhere; 2. pick your platform; 3. work through Crawl, Walk and Run.

Read alongside: The AI Universe · Agent Harness, drawn simply · The Agent Harness Series · Eval Harness, drawn simply · Vertical agents, by industry · Certification prep on this site.

Step 1 · the same everywhere

Key concepts

Learn these ideas first. They are the same on every cloud and with every tool; only the product names change. Each one is a short page: what it is, why it matters, an example and the common mistake.

Step 2

Where do you work?

Pick your platform and you get one page with that platform's products in every step.

Beginner · 6 weeks

Crawl: You can build one thing end to end, with help.

Call a model, get checked structured output, and give it a few safe tools.

Key ideas at this level, the same on every platform:

Open this level on: AWS · Azure · Google Cloud · Databricks · Open source

Learn: the concept, and the product on each platform

LearnAWSAzureGoogle CloudDatabricksOpen source
Python and APIsHTTP, JSON, async, tests; read an API's docsPython
Lambda / ECS Fargate
Python
Container Apps / Functions
Python
Cloud Run
Python
Databricks Apps
Python
Docker
Calling a modelmessages, tokens, temperature, cost per callAmazon BedrockAzure OpenAI in Microsoft FoundryGemini on Vertex AIFoundation Model APIsOllama, or a free hosted open model
Structured outputask for JSON, validate it, retry on failureBedrock Converse APIFoundry structured outputsGemini controlled generationai_query + structured outputPydantic + JSON mode
Toolslet the model ask your code to act; check before actingBedrock tool useFoundry function callingGemini function callingUnity Catalog functions as toolsplain Python loop, no framework
Deterministic firstif code can do it, code does it; the model handles the restPythonPythonPythonPythonPython

Build on GitHub

Under way · join

Agent harness from scratch

Build the loop, tools, context and checks around a model in plain Python, one small step at a time.

Proposed · needs a lead

Your first agent, three tools

A support assistant that can look up an order, check a policy and draft a reply, where code checks every tool call first.

You are ready to move on when

Intermediate · 3 months

Walk: You can build it so it keeps working, without help.

Build the harness: retrieval, state, approvals, tracing and evaluation on every change.

Key ideas at this level, the same on every platform:

Open this level on: AWS · Azure · Google Cloud · Databricks · Open source

Learn: the concept, and the product on each platform

LearnAWSAzureGoogle CloudDatabricksOpen source
Retrievalsearch by meaning; chunking; citationsOpenSearch Serverless / Aurora pgvectorAzure AI SearchVertex AI Vector Search / AlloyDBVector Searchpgvector or Qdrant
Agent loop and statesteps, budgets, checkpoints, waiting for a humanBedrock AgentCoreFoundry Agent Service / Durable FunctionsVertex AI Agent EngineMosaic AI Agent FrameworkDBOS or Temporal + your loop
Checks and guardrailsbefore and after every model and tool callBedrock GuardrailsAzure AI Content SafetyModel ArmorAI Gateway guardrailsrule checks in your own code
Tracingevery call, token and cent visibleCloudWatch + X-Ray (OpenTelemetry)App Insights (OpenTelemetry)Cloud Trace (OpenTelemetry)MLflow tracingPhoenix or Langfuse
Evaluationgolden cases, rules, a gate in CIBedrock evaluations
GitHub Actions + CodeDeploy
Foundry evaluations
GitHub Actions or Azure Pipelines
Vertex AI Gen AI evaluation
GitHub Actions + Cloud Build
MLflow evaluation
GitHub Actions + bundles
Eval Harness Framework or promptfoo
GitHub Actions

Build on GitHub

Under way · join

Claims assistant on Azure

A motor-insurance claims agent running for real: tools, checks, approvals, durable waits, evaluation and cost.

Under way · join

LearnLangGraph

A hands-on course in graph-shaped agents: state, branches, human approval and resuming after a crash.

Under way · join

Reference agent

A customer-support agent with no framework: one module per harness layer, the architecture enforced by tests.

You are ready to move on when

Advanced · 6 months +

Run: You can design it, prove it and lead others through it.

Prove an agent before it meets real people, and run many of them at a known cost.

Key ideas at this level, the same on every platform:

Open this level on: AWS · Azure · Google Cloud · Databricks · Open source

Learn: the concept, and the product on each platform

LearnAWSAzureGoogle CloudDatabricksOpen source
Simulated worldstest against simulated customers, systems and faultsBedrock evaluationsFoundry evaluationsVertex AI Gen AI evaluationMLflow evaluationEval Harness Framework or promptfoo
Gateway and budgetsmany models behind one door; fallbacks; spend limitsBedrock cross-region inference + API GatewayAPI Management AI gatewayApigeeMosaic AI GatewayLiteLLM
Assurancewhat must be true before it is trusted; evidence, not claimsBedrock evaluations
Bedrock Guardrails
Foundry evaluations
Azure AI Content Safety
Vertex AI Gen AI evaluation
Model Armor
MLflow evaluation
AI Gateway guardrails
Eval Harness Framework or promptfoo
rule checks in your own code
Portabilitysame agent, new cloud, by swapping adaptersTerraform or AWS CDK
Bedrock AgentCore
Bicep or Terraform
Foundry Agent Service / Durable Functions
Terraform
Vertex AI Agent Engine
Declarative Automation Bundles (Asset Bundles)
Mosaic AI Agent Framework
Docker Compose
DBOS or Temporal + your loop
Researchlong-running agents, self-repair, driftCloudWatch + X-Ray (OpenTelemetry)App Insights (OpenTelemetry)Cloud Trace (OpenTelemetry)MLflow tracingPhoenix or Langfuse

Build on GitHub

Under way · join

Eval Harness Framework

Rebuild each conversation from the agent's own traces and check it against rules written in plain YAML.

Under way · join

AgentTwin

Simulate the agent's world (customers, systems, faults, time) so a real agent can be tested before it meets real people.

Proposed · needs a lead

Port the agent to your cloud

Take the reference agent and bind it to AWS or Google Cloud: same code, new adapters, same tests passing.

Under way · join

AgentTwin Lab

Research: an agent runs for weeks in a simulated business while a fixer repairs it. Does the fixer cheat?

You are ready to move on when