Home › On your cloud › Open source
On your cloud · the same headings as the concepts
Open source
The same platform built from open-source parts: Spark, Delta or Iceberg, dbt, Airflow or Dagster, OpenMetadata. The headings match the concepts tree: learn the idea in the concepts, then see here which Open source product does each job. Product names change often; check the current docs before you build.
Open source
Data systems: in and out
| Job | Product on Open source |
|---|---|
| Change data capture from databases | Debezium |
| Connectors to SaaS apps | Airbyte or dlt |
| Sharing data out | Delta Sharing (open protocol) |
No pages for this on Open source yet; the concepts apply as they are.
Open source
Data engineering
| Job | Product on Open source |
|---|---|
| Storage | MinIO or local disk |
| Table format | Delta Lake or Iceberg |
| Into landing | dlt or Airbyte |
| Landing → bronze | PySpark |
| Bronze → silver → gold | PySpark or dbt + DuckDB |
| Quality checks | Great Expectations or Soda |
| Catalog, access, lineage | OpenMetadata + OpenLineage |
| Semantic layer | Cube Core |
| Orchestration | Airflow or Dagster |
Shift left: fail fast on data code
Open checks for PySpark, SQL and Databricks code.
Open source
BI and analytics
| Job | Product on Open source |
|---|---|
| Semantic layer | Cube Core |
| Dashboards | Apache Superset |
| Ask in plain words | Your own, over Cube |
No pages for this on Open source yet; the concepts apply as they are.
Open source
ML
| Job | Product on Open source |
|---|---|
| Experiments and model registry | MLflow |
| Features | Feast |
| Model serving | KServe or BentoML |
No pages for this on Open source yet; the concepts apply as they are.
Open source
AI
| Job | Product on Open source |
|---|---|
| Models | Open-weight models served with vLLM |
| Agents | LangGraph, or your own harness |
| Retrieval | pgvector or Qdrant |
| Evaluation and guardrails | Eval Harness Framework (this site) |
No pages for this on Open source yet; the concepts apply as they are.
Open source
Across all of them
| Job | Product on Open source |
|---|---|
| Governance and access | OpenMetadata |
| Cost | Your own cost tables |
| Observability | OpenTelemetry |
| Infrastructure as code | Terraform or OpenTofu |
Open-source code from this site
Reference Agent, AgentTwin, Eval Harness Framework and more.
Open source
Learning paths by role
Crawl, walk and run for each role, on Open source.
Open source
Certifications
No free prep on this site yet. See all certifications.