Home › Paths › AI Engineering › Open source

Learning path · Open source

AI Engineering on Open source

You build products and agents where a model does part of the work, safely.

Free, like everything here: no sign-up, no paywall. Only the cloud you practise on may charge; the open-source path costs nothing.

Most of an AI system is ordinary software around a small model call: the tools, the checks, the state, the evaluation and the bill. AI engineers build that harness so the product holds up after the demo.

Every topic below is a concept (the job) with the Open source product that does it. Learn the concept once; on another platform only the product name changes. Read alongside: The AI Universe · Agent Harness, drawn simply · The Agent Harness Series · Eval Harness, drawn simply · Vertical agents, by industry · Certification prep on this site.

Beginner · 6 weeks

Crawl: You can build one thing end to end, with help.

Call a model, get checked structured output, and give it a few safe tools.

Key ideas at this level, the same on every platform:

Learn

LearnWhat it meansOn Open source you use
Python and APIsHTTP, JSON, async, tests; read an API's docsPythona language, properly
Dockerrun an app
Calling a modelmessages, tokens, temperature, cost per callOllama, or a free hosted open modelmodel api
Structured outputask for JSON, validate it, retry on failurePydantic + JSON modeprompts and structured output
Toolslet the model ask your code to act; check before actingplain Python loop, no frameworktool calling
Deterministic firstif code can do it, code does it; the model handles the restPythona language, properly

Build on GitHub

Under way · join

Agent harness from scratch

Build the loop, tools, context and checks around a model in plain Python, one small step at a time.

Built on Open source already.

Proposed · needs a lead

Your first agent, three tools

A support assistant that can look up an order, check a policy and draft a reply, where code checks every tool call first.

You are ready to move on when

Intermediate · 3 months

Walk: You can build it so it keeps working, without help.

Build the harness: retrieval, state, approvals, tracing and evaluation on every change.

Key ideas at this level, the same on every platform:

Learn

LearnWhat it meansOn Open source you use
Retrievalsearch by meaning; chunking; citationspgvector or Qdrantvector search
Agent loop and statesteps, budgets, checkpoints, waiting for a humanDBOS or Temporal + your loopagent runtime
Checks and guardrailsbefore and after every model and tool callrule checks in your own codeguardrails
Tracingevery call, token and cent visiblePhoenix or Langfusetracing
Evaluationgolden cases, rules, a gate in CIEval Harness Framework or promptfooevaluation
GitHub Actionsci/cd

Build on GitHub

Under way · join

Claims assistant on Azure

A motor-insurance claims agent running for real: tools, checks, approvals, durable waits, evaluation and cost.

Built on Azure today. Porting it to Open source is a good first contribution.

Under way · join

LearnLangGraph

A hands-on course in graph-shaped agents: state, branches, human approval and resuming after a crash.

Built on Open source already.

Under way · join

Reference agent

A customer-support agent with no framework: one module per harness layer, the architecture enforced by tests.

Built on Open source already.

You are ready to move on when

Advanced · 6 months +

Run: You can design it, prove it and lead others through it.

Prove an agent before it meets real people, and run many of them at a known cost.

Key ideas at this level, the same on every platform:

Learn

LearnWhat it meansOn Open source you use
Simulated worldstest against simulated customers, systems and faultsEval Harness Framework or promptfooevaluation
Gateway and budgetsmany models behind one door; fallbacks; spend limitsLiteLLMmodel gateway
Assurancewhat must be true before it is trusted; evidence, not claimsEval Harness Framework or promptfooevaluation
rule checks in your own codeguardrails
Portabilitysame agent, new cloud, by swapping adaptersDocker Composeinfrastructure as code
DBOS or Temporal + your loopagent runtime
Researchlong-running agents, self-repair, driftPhoenix or Langfusetracing

Build on GitHub

Under way · join

Eval Harness Framework

Rebuild each conversation from the agent's own traces and check it against rules written in plain YAML.

Built on Open source already.

Under way · join

AgentTwin

Simulate the agent's world (customers, systems, faults, time) so a real agent can be tested before it meets real people.

Built on Open source already.

Proposed · needs a lead

Port the agent to your cloud

Take the reference agent and bind it to AWS or Google Cloud: same code, new adapters, same tests passing.

Under way · join

AgentTwin Lab

Research: an agent runs for weeks in a simulated business while a fixer repairs it. Does the fixer cheat?

Built on Open source already.

You are ready to move on when

Optional

Certifications on Open source

Projects prove more than certificates, but a certificate can open a door. These fit this path. Exam names and codes change, so check the current one before you book.

Free prep we have written: Certifications.

Learn together

Do it with others

Every project above has a page, a list of open work and a "good first issue" label on GitHub. Pick one at your level, say hello by email, and build it in the open. If a project says needs a lead, that can be you.

All projects →   Another path →