Home › On your cloud › Open source

On your cloud · the same headings as the concepts

Open source

The same platform built from open-source parts: Spark, Delta or Iceberg, dbt, Airflow or Dagster, OpenMetadata. The headings match the concepts tree: learn the idea in the concepts, then see here which Open source product does each job. Product names change often; check the current docs before you build.

Open source

Data systems: in and out

JobProduct on Open source
Change data capture from databasesDebezium
Connectors to SaaS appsAirbyte or dlt
Sharing data outDelta Sharing (open protocol)

No pages for this on Open source yet; the concepts apply as they are.

Open source

Data engineering

JobProduct on Open source
StorageMinIO or local disk
Table formatDelta Lake or Iceberg
Into landingdlt or Airbyte
Landing → bronzePySpark
Bronze → silver → goldPySpark or dbt + DuckDB
Quality checksGreat Expectations or Soda
Catalog, access, lineageOpenMetadata + OpenLineage
Semantic layerCube Core
OrchestrationAirflow or Dagster

Shift left: fail fast on data code

Open checks for PySpark, SQL and Databricks code.

Open source

BI and analytics

JobProduct on Open source
Semantic layerCube Core
DashboardsApache Superset
Ask in plain wordsYour own, over Cube

No pages for this on Open source yet; the concepts apply as they are.

Open source

ML

JobProduct on Open source
Experiments and model registryMLflow
FeaturesFeast
Model servingKServe or BentoML

No pages for this on Open source yet; the concepts apply as they are.

Open source

AI

JobProduct on Open source
ModelsOpen-weight models served with vLLM
AgentsLangGraph, or your own harness
Retrievalpgvector or Qdrant
Evaluation and guardrailsEval Harness Framework (this site)

No pages for this on Open source yet; the concepts apply as they are.

Open source

Across all of them

JobProduct on Open source
Governance and accessOpenMetadata
CostYour own cost tables
ObservabilityOpenTelemetry
Infrastructure as codeTerraform or OpenTofu

Open-source code from this site

Reference Agent, AgentTwin, Eval Harness Framework and more.

Open source

Learning paths by role

Crawl, walk and run for each role, on Open source.

Open source

Certifications

No free prep on this site yet. See all certifications.