Home › Learn › Databricks compute and cost › Backup architecture
Databricks compute and cost · data engineering
Backup and recovery architecture for Databricks: the safe, clean design on any cloud
Time travel is not a backup. Three protections for three disasters: fast undo, an immutable backup in a separate account, and a rebuildable second region, with the design rules and a failure-to-tool table.
Prices, defaults, limits and preview status change often, and dollar figures here are illustrations. Check the current Databricks documentation and price list before quoting a number.
Backup and recovery
Best architecture for backup and recovery
Start with the surprise: Delta time travel is not a backup. It lives in the same storage as the table, and VACUUM deletes old files by design (history is kept for a limited time, often a week). Time travel is a fast undo. A backup is a separate copy that an attacker or a bad script cannot touch.
| Failure | Tool | Typical recovery speed | Verdict |
|---|---|---|---|
| Bad write, bad MERGE, wrong job run | Delta time travel and RESTORE TABLE | Minutes | Pursue as first line |
| Table dropped by mistake | UNDROP TABLE (managed tables, limited window) or object versioning | Minutes to hours | Pursue |
| Files lost, vacuumed early or corrupted | Incremental deep clone in a separate location | Hours | Pursue |
| Malicious delete, ransomware, compromised admin | Backup in a separate account with immutability and no delete rights for production | Hours | Pursue for critical data |
| Region outage | Secondary region with replicated data and Terraform-built workspace | Hours (warm) to a day (cold) | Pursue only for what the business needs. It is expensive. |
| Workspace or job configuration lost | Terraform, Asset Bundles, Git | Hours | Pursue |
| Copying the raw bucket between regions with plain storage replication, for Delta tables | Raw replication can arrive out of order, so a table may point at files that have not arrived yet | Risk of a broken table | Drop for Delta. Use deep clone, or validate after replication. |
The design rules
- Decide RPO and RTO per data set. RPO is how much data you can afford to lose, RTO is how long you can be down. Most tables do not need DR. A few do.
- Everything that can be code is code. Workspaces, catalogs, grants, jobs, warehouses and policies in Terraform or Asset Bundles, kept in Git.
- Backups live in a different account with different credentials. Production may write, but never delete. Turn on immutability.
- Use Delta-aware copies. Deep clone is incremental, copies data and log together, and is consistent.
- Unity Catalog metadata is in the control plane. Back up its definitions (catalogs, schemas, grants, external locations) through Terraform or API exports, because the data copy alone does not recreate permissions.
- Practise the restore. A backup you have never restored is a hope. Schedule a restore drill and time it.
- Mind the bill. Cross-region copies cost transfer fees (see cross-region cost) and storage. Back up gold and critical tables, not every scratch table.
FAQ
Questions people ask
Is Delta time travel a backup?
No. It lives in the same storage and VACUUM deletes old files by design. It is a fast undo.
Is copying the bucket to another region safe for Delta tables?
Not by itself. Files can arrive out of order, so a table's log may point at files that have not arrived. Use deep clone or validate after replication.
Next
Where to go next
Related on this site: Backup and recovery on Azure · Cheapest way to move data in and out
All Databricks compute and cost pages · All data engineering · Databricks on this site