Home › Learn › Databricks interview prep › Egress and ingress

Databricks interview prep · data engineering

Cheapest way to move data in and out

The cheapest way to egress and ingress data on Databricks: avoid, shrink, then pick the path, with hidden network costs and a way to find them.

Maintenance commandsRow-level securityCompute choiceJoinsSkew and shuffleAQE, Photon, POCost and performance leversDeclarative and semanticAsset BundlesLakeflow ConnectUniFormVector Searchdbt and ODCSEgress and ingressMaintenance in picturesClustering and compactionMaintenance strategySlow writesBackup and recoveryTaggingQuestion bank

Defaults, limits and preview status change often. Check the current Databricks docs before quoting a number.

Don't move itMove less of itMove it by the cheapest path

The cheapest transfer is the one you never do.

Egress

The cheapest way to move data out

Prices vary by cloud, region and contract. Check your provider's price sheet before quoting a number.

Cloud providers charge for data leaving a region or the cloud and charge little or nothing for movement inside the same region.

StepOptions
Do not move itRun compute in the same cloud and region as the storage; query in place and return only results; use Delta Sharing for live reads; move the consumer to the data. Cross-region sharing still incurs egress; Databricks supports a Cloudflare R2 replica for sharing, because R2 has no egress fees (check current support).
Move lessFilter and aggregate first; select only needed columns; compress (Parquet with zstd or snappy); send increments with change data feed; export columnar not CSV or JSON; cache on the consumer side; cap result sizes so nobody pulls a whole table to a laptop.
Cheapest pathSame region is usually free; cross-region is cheaper than internet; private circuits (Direct Connect, ExpressRoute, Interconnect) have lower per-GB rates but a fixed fee, so they pay only at steady high volume; free-egress stores or a CDN for widely downloaded data; physical transfer devices for one-off petabyte moves; negotiated discounts at scale.
Hidden egress costs in DatabricksDetail
Cross-AZ trafficShuffle between zones is charged by some clouds
NAT gateway processingPer-GB fees can exceed the transfer itself; use storage gateway endpoints
Cross-region storage readsEvery query from a workspace in another region is egress
Disaster recovery replicationContinuous cross-region copies are an ongoing bill; replicate critical tables incrementally
Federation queriesPull data from external databases, possibly across regions
Result downloads and exportsFrom notebooks and warehouses to local machines

To find the source: use the cloud cost tool filtered on data-transfer lines by region pair, storage access or flow logs, and Databricks system tables (billing, audit, query history). Fix the top one or two sources first.

Ingress

The cheapest way to bring data in

Inbound transfer to AWS, Azure and Google Cloud is generally free. The cost is the source's egress, the network path you build, the compute that loads, and storage requests.

SourceCheapest approach
Files in cloud storage, same cloudAuto Loader, or COPY INTO for simple batch
Files in another region or cloudCopy once in bulk, compressed, then ingest locally
Operational databaseLog-based CDC (Lakeflow Connect where supported) or incremental queries on an indexed watermark; avoid full reloads
SaaS appsManaged connector if supported, else scheduled incremental API pulls using bulk endpoints
Event streamsStructured Streaming; availableNow batches if freshness allows
On-premises dataCompress and upload in bulk; transfer devices for huge one-offs
Another lakehouse or warehouseFederation or Delta Sharing, no copy
Big historical backfillOne run on a right-sized job cluster with spot workers
TechniqueEffect
Incremental loadsOnly new or changed data
Compressed columnar filesParquet beats CSV and JSON
Files of 128 MB to 1 GBMillions of tiny files raise request costs and slow Spark
Land raw, transform laterCheap, rerunnable ingest job
Scheduled instead of continuousavailableNow runs when latency allows
Serverless or job clusters with auto-terminationPay only while loading
Spot workersFault-tolerant batch ingestion
File notification mode at large scaleAvoids repeated listing of huge directories
Lifecycle rules on the landing zoneMove old raw files to cheaper tiers

Hidden costs: NAT gateway processing, private link hourly and per-GB fees, dedicated circuits, cross-AZ and cross-region traffic, third-party ELT tools that charge by rows, request charges, source load, and failed reruns that reload everything (use checkpoints and idempotent writes).

Practice

Questions to rehearse

  1. How would you reduce egress cost?
  2. Is Delta Sharing free of egress?
  3. What hidden costs come with private networking?
  4. What does ingress cost, really?
  5. How do you ingest cheaply at scale?
  6. When would you not ingest at all?
In an interview, say: Co-locate compute and storage, shrink what moves, make transfers incremental, and watch the hidden network fees. Inbound transfer is free; the source, network path and compute are not.

FAQ

Questions people ask

Is Delta Sharing free of egress?

No. Data read across regions or clouds is still billed, unless it sits in a store with no egress fee.

Why is a NAT gateway a risk?

It charges per GB processed, which can exceed the transfer cost; storage endpoints avoid it.

Next

Where to go next

All Databricks interview topics · All data engineering · Databricks on this site