Learn

Free, picture-first courses for data and AI engineering, from the first drawing to the last line of code. Looking for a step-by-step plan instead? Pick your path.

Learn

The agent harness, end to end

One real agent, a motor-insurance claims assistant running on Azure, followed from the moment a customer types to the moment the reply goes back. Every box in every drawing opens the next level down, until you reach the real code.

The AI Universe

start here · the map before the trek

Draw the map first. Seven families of stacks, the concept buckets inside each, and the products that fill them on open source, Azure, AWS and GCP. Learn the concept once; place every new name in a minute.

Agent Harness, drawn simply

tutorial · 167 pages · hand-drawn

Start here. Picture-first: the architecture, one message end to end, the loop, context, checks, state, durable waits, eval, cost and testing. Call stacks, class diagrams and real code on every page, plus 19 real flows and 20 tech comparisons.

Agent Harness Simulator

hands-on · type a message, watch every step

Try it. Send a message to the claims assistant and follow the numbered steps on a map: every system it talks to, every check that fires, the code, the variables, and the JSON to and from the model. Then break it: switch the agent off, plant an injection, take the model down. See what reaches App Insights and Prometheus, and what the evaluations say.

The agent guardrail catalog

reference · 57 guardrails · filterable

Every common guardrail, sorted. Where each one sits in a turn, must or should have, whether a rule, a classifier, an LLM judge or a person decides, what it does when it fires, example tools, and the test that proves it works.

Running AI you can trust

guide · 18 capabilities · one page

Governance, safety, security and reliability across a whole organization. For each of eighteen capabilities: the leadership question, what to decide and build, open standards and tools, how to make it better, how to prove it works, and how much can be automated.

AgentTwin, explained

guide · step-through simulator · honest status

How do you know an agent is ready? AgentTwin keeps the agent real and simulates its world: customers, systems, data, approvers, faults and time, then judges what changed. AgentTwin vs a digital twin, how a given agent's tests are chosen, a tiny agent you can play, coverage of the Assurance Catalog, and how it differs from LangWatch and others.

The technical anchor

guide · data engineering · for developers stepping up

How to think for a whole data project. The twelve roles around you and what each needs, failing fast on architecture, estimates, scope and decisions, helping stakeholders find their mistakes early, risks, and test-deploy-run from week one. With a first-30-days plan.

DataTwin, explained

guide · step-through simulator · honest status

Is the number on the dashboard right? DataTwin keeps the data platform real and simulates the business that feeds it: a shop that keeps selling, its source systems, and a calendar of planted incidents. Each simulated day the dashboard is checked against the world's truth. The data-engineering sibling of AgentTwin: designed, with the learning studio built.

The Agent Harness Series

deep dive · 22 concerns · Azure

Every detail. The long version: each concern on its own page, the Azure architecture, real traces and payloads from live turns, and the design decisions with their trade-offs.

Eval Harness, drawn simply

tutorial · 40 pages · built alongside the code

Checking what an agent did. How an open-source framework rebuilds each conversation from the agent's own traces and checks it against rules written in plain YAML: was the claim read before it was paid, did it say “paid” when nothing was paid. Every example on the pages is the real code's output.

Data engineering

The data platform, end to end

How data moves from the systems that make it to the dashboards and AI that use it, and the rules that keep the numbers right. More guides and a simulator are on the way.

Data systems, end to end

guide · for data leaders first · drill down hop by hop

Upstreams, the data platform, downstreams. One picture, then a page per block and per hop: the questions to chase and why, what breaks in production if nobody asks, and the decisions with a verdict. Starts with the sources: push, pull or share, how rows change, keys and dates, and why we ask to profile your data.

Data platform architecture, drawn right

guide · the whole picture · click any box

Start with the map. The control plane and metadata, three services, the medallion layers, streaming, 22 cross-cutting concerns, and agents that may only suggest. Every box opens the next level down.

DataTwin Studio

simulation · recorded from a real run · break it

Watch one batch go through the platform. Eight panels: config and metadata, Spark jobs and tasks, rows in every layer, the Delta Lake commit files, the query plan, logs and the dashboard. Then send the same file again: our pipeline stays right, a naive one doubles the revenue.

Medallion architecture, drawn right

guide · bronze, silver, gold · every platform

Zoom into the middle row. Each layer's job and promise, one order followed through every layer with real rows, nine rules, the common mistakes, and the products on AWS, Azure, Google Cloud, Databricks and open source.

Real-time analytics, drawn simply

guide · 11 short pages · abstract first, then products

From a database change to a number on screen in a second. Real time vs streaming vs near real time, Lambda vs Kappa, the components left to right, change capture tools by database, when Kafka is needed, eight designs and their products on each cloud, push to the screen, and cost.

Hard problems by layer, and how to solve them

guide · 10 layers and hops · 69 challenges, filterable

From the source system to the screen, the hard parts of each layer and the common mistakes. Why silver to gold and gold to semantic are hardest, a worked tyre-company example for every challenge, what is solved, assisted or left to people, and the anti-patterns to avoid.

Shift left: fail fast on data code

guide · 10 short pages · open-source tools, no servers

Catch problems in PySpark, Python and Databricks SQL before the code runs. Pure functions and how to detect them, unit tests and coverage, SCD2, idempotency, duplicates and dropped rows tested on a laptop, complexity scores, clean-code rules, security and cost checks, a gap report for any repo, and why SonarQube isn't needed.

Key concepts

25 data engineering ideas · one short page each

The ideas that are the same everywhere. ETL vs ELT, file formats, Delta and Iceberg, partitioning, SCD, data contracts, lineage, idempotency and more.

Certifications

Exam prep

Study guides, practice by domain and mock exams for the Claude Certified Architect and Microsoft AI-103 exams.

Open certification prep →