Home › Learn › Databricks interview prep › Lakeflow Connect
Databricks interview prep · data engineering
Lakeflow Connect: managed ingestion
What Lakeflow Connect is, how it works, when to use it versus Auto Loader, federation, sharing and COPY INTO, and the hard points to know.
Defaults, limits and preview status change often. Check the current Databricks docs before quoting a number.
A managed copy from a source into your lakehouse, kept up to date.
The idea
What it is
Lakeflow Connect is a set of managed ingestion connectors. You give a connection and a target; Databricks runs the pipeline that copies the data into Unity Catalog Delta tables and keeps it current. You write no extraction code and run no CDC infrastructure. For databases it reads the change log, so inserts, updates and deletes are captured. It runs on serverless pipeline compute and handles schema changes. It can also be set up in the UI, the API, or as a bundle.
The connector list and status change often (SaaS apps such as Salesforce, Workday, ServiceNow and Google Analytics; databases such as SQL Server; SharePoint). Check the current list.
Choose
When to use it and when not
| Situation | Choice |
|---|---|
| Supported SaaS app, no wish to maintain API code | Lakeflow Connect: pagination, rate limits, retries and schema changes handled |
| Continuous replication from an operational database | Lakeflow Connect: log-based CDC, low source load, deletes captured |
| Small team, many sources | Lakeflow Connect: one pattern instead of many jobs |
| Files landing in cloud storage | Auto Loader |
| Message streams (Kafka, Event Hubs, Kinesis) | Structured Streaming connectors |
| Source not supported | Custom code or a third-party tool |
| Occasional queries, small data, freshness not critical | Lakehouse Federation (query in place) |
| Data already shared by another platform | Delta Sharing |
| Simple idempotent batch file load | COPY INTO |
| Heavy transformation during extraction | Ingest raw first, transform in pipelines |
| Private source the connector cannot reach | Check private connectivity first |
| Replacing a third-party ELT tool | Compare real costs for the same sources |
Hard points
Questions behind the features
- Deletes: log-based CDC captures them; SaaS connectors depend on what the app exposes. Ask per source.
- Schema changes: the connector adapts to many, but downstream contracts must handle a new column.
- First load: a large first snapshot can be heavy; plan for source load and API quotas.
- Source setup: log-based CDC needs the source configured (for example change tracking); the database owner must agree.
- Cost: serverless compute for pipeline and gateway; watch it in system tables.
- Latency: near real time for databases; SaaS sources are limited by API rate and polling.
- Medallion: land as bronze; no business logic in ingestion.
- Recovery: know how a failed run and a full refresh behave.
Practice
Questions to rehearse
- Lakeflow Connect versus Auto Loader: how do you choose?
- Connect versus Lakehouse Federation: when do you copy and when query in place?
- How does log-based CDC avoid hammering the source?
- How are deletes propagated into the Delta target?
- The source adds a column. What happens?
- How would you justify replacing Fivetran with it?
FAQ
Questions people ask
Is it the same as Auto Loader?
No. Auto Loader ingests files from storage. Connect copies from apps and databases.
Does it capture deletes?
Database connectors read the change log, so yes. SaaS depends on the source's API.
Next
Where to go next
All Databricks interview topics · All data engineering · Databricks on this site