Home › Learn › Databricks compute and cost › Cost by workload

Databricks compute and cost · data engineering

Databricks cost map: every workload type, every compute type, DBU rates, the mapping and what goes wrong

One table of workload types, one of compute types with illustrative DBU rates, and the mapping between them with worked monthly examples, the wrong-compute multipliers and a monthly check.

Cluster anatomyWarehouses and serverlessChoosing computeCompute settingsCost by workloadLogs and visibilityPhoton and cachesHow billing worksDividing the billSystem tablesSharing and regionsBackup architectureCost-saving hacks

Prices, defaults, limits and preview status change often, and dollar figures here are illustrations. Check the current Databricks documentation and price list before quoting a number.

How to read this page

Workloads, compute, rates, and the mapping between them

A Databricks bill is DBUs used × price per DBU, plus your cloud's machine bill on classic compute. Both parts depend on which compute you chose, so the same work can cost very different amounts. This page has four tables: every kind of workload, every kind of compute with its rate, the mapping between them with a monthly example, and what goes wrong in that mapping.

All dollar figures are illustrations. The rates are published AWS Premium list prices in a US region, as summarised by several third-party pricing guides. I did not check the official price list. Azure and Google Cloud differ (one source lists Azure Premium jobs compute at roughly double the AWS figure), contract discounts change the numbers, and DBU rates change. The machine assumptions (2 DBU per hour and $0.384 per hour per machine) are the same ones used on the billing page. Use the tables to compare ratios and mistakes, then redo the arithmetic with your own price list and your own system tables.

Table 1

Every kind of workload

#WorkloadTypical patternConcurrencyLatency needSpecial needs
W1Nightly batch ETL1 to 3 hours, once a dayOne runDone by morningNone
W2Large steady ETLMany hours a day, large dataSeveral runsHoursSpot pricing, Scala or RDD code
W3Backfill or one-time loadDays, onceOne runNoneLarge, short-lived
W4Ad-hoc SQLA few analysts, bursts through the day1 to 5 queriesSeconds to startNone
W5Dashboards and Power BIMany users, scheduled and live refreshTens of queriesSeconds per queryCost attribution per team
W6Notebook developmentHours a day, one user per notebookOne userQuick startLibraries
W7ML trainingHours, burstyOne or a fewNoneGPU, RDD, R
W8Always-on streaming24 hours a day, 7 days a weekContinuousSecondsContinuous trigger
W9Micro-batch streamingRuns every 5 to 15 minutes, finishes quicklyShort runsMinutesFast start-up
W10Declarative pipelinesTriggered on a scheduleOne updateMinutes to hoursNone
W11Live application queriesAlways on, many small queriesConstantSub-second to secondsLoad test needed
W12Admin and cost queriesA few queries a day against system tables1 to 2Not urgentSmall and cheap

Table 2

Every kind of compute and what it costs per DBU

#ComputeWhere it runsIllustrative rate per DBUMachines in the rate?Idle and start-up
C1Jobs compute (classic job cluster)Your accountabout $0.15No, plus your cloud VMsYou pay for boot time. The cluster ends with the run.
C2Serverless jobsDatabricksabout $0.35 (more for some modes and plans)YesStart-up time is not billed. Standard mode uses fewer DBUs and starts slower.
C3All-purpose cluster (classic)Your accountabout $0.55No, plus your cloud VMsBills until it terminates, including idle time
C4Serverless notebooks (all-purpose serverless)Databricksabout $0.75YesIdle auto-termination
C5Classic SQL warehouseYour accountabout $0.22No, plus your cloud VMsMinutes to start. Default auto-stop 45 minutes.
C6Pro SQL warehouseYour accountabout $0.55No, plus your cloud VMsMinutes to start. Default auto-stop 45 minutes.
C7Serverless SQL warehouseDatabricksabout $0.70YesSeconds to start. Default auto-stop 10 minutes.
C8Pipelines, classicYour accountabout $0.20, $0.25 or $0.36 by editionNo, plus your cloud VMsTriggered mode ends. Continuous stays up.
C9Serverless pipelinesDatabricksCheck the price listYesTriggered mode, no cluster to idle
C10Other serverless features (Predictive Optimization, model serving, vector search)DatabricksOwn rate. Predictive Optimization bills on a serverless jobs SKU.YesVaries. Some endpoints bill while they exist.

How many DBUs per hour

Table 3

The mapping: workload to compute, with a monthly example

Each example uses the assumptions above. A cluster with 1 driver and 4 workers is 5 machines, so 10 DBU per hour and $1.92 per hour in cloud VMs.

WorkloadRight computeMonthly cost (example)Same work on the wrong computeVerdict
W1 Nightly ETL, 2 hours a day, 60 hours a month, 5 machinesC1 jobs compute600 DBU × $0.15 = $90, plus VMs $115.20 = $205.20C3 all-purpose: 600 × $0.55 = $330, plus VMs $115.20 = $445.20, about 2.2 times morePursue C1, or C2
W1 on serverlessC2 serverless jobs600 DBU × $0.35 = $210, no VM line. In Standard mode, Databricks cites up to 70 percent fewer DBUs, which would be about $63.On paper close to C1. Differences come from boot time on classic and from start-up delay in Standard mode.Pursue C2 for simplicity
W5 Dashboards on a Medium warehouse (24 DBU per hour), 22 daysC7 serverless, 4 busy hours a day: 88 h × 24 = 2,112 DBU × $0.70 = $1,478The same warehouse kept awake 10 hours a day by refreshes: 220 h × 24 = 5,280 DBU × $0.70 = $3,696, which is $2,218 moreC6 pro: $13.20 per hour in DBUs, so $1,162 for 88 busy hours, plus VMs and the 45-minute idle tails. C5 classic: $5.28 per hour, so $465, plus VMs.Pursue C7 with short auto-stop
W4 Ad-hoc SQL on a Small warehouse (12 DBU per hour), about 5 sessions a dayC7 serverless: 66 busy hours plus about 18 hours of 10-minute idle tails = 84 h × 12 = 1,012 DBU × $0.70 = $708C6 pro with the default 45-minute stop: 66 busy plus 82 idle hours = 148 h × 12 = 1,782 DBU × $0.55 = $980 in DBUs, plus VMs for 148 hoursPursue C7
W6 Notebook work on a 4-machine all-purpose cluster (8 DBU per hour, $1.536 per hour in VMs)C3 with auto-terminate, 4 active hours a day: 88 h × 8 = 704 DBU × $0.55 = $387, plus VMs $135 = $522The same cluster left running 8 hours a day: 176 h → $774 + $270 = $1,045, doublePursue short auto-terminate, or C4
W8 Always-on streaming, 3 machines (6 DBU per hour), 730 hoursC1 jobs compute, fixed size: 4,380 DBU × $0.15 = $657, plus VMs $841 = $1,498C3 all-purpose by mistake: 4,380 × $0.55 = $2,409, plus VMs $841 = $3,250. C2 serverless cannot run continuous triggers.Pursue C1
W9 Micro-batch every 15 minutesC2 serverless jobs, because a classic cluster would boot every runDepends on DBUs per run. Compare DBU per run over time.A classic job cluster pays boot minutes 96 times a day. An always-on cluster pays all day.Pursue C2
W10 Declarative pipelinesC9 serverless, or C8 classic in triggered modeRate from the price list, times DBUs per updateContinuous mode keeps compute running around the clockPursue triggered
W2 Large steady ETL with spotC1 with spot workersThe same DBU part as W1, but the VM part shrinks by the spot discountOn C2 you pay the serverless rate for every hour, and cannot use spotPursue C1 and price both
W7 ML with GPU or RDDC3 dedicated, with a GPU node typeMachine-dominated. GPU VMs often cost more than the DBUs.Serverless notebooks restrict RDD and Spark context. I did not confirm GPU options.Pursue C3 dedicated
W12 Admin and system-table queriesC7 smallest size, shortest stopSeconds of a 2X-Small (4 DBU per hour) a day. Roll up once instead of querying often.A large pro warehouse kept running for dashboards over system tablesPursue
W11 Live application queriesA warehouse sized for the peak, after a load test, or a database built for point lookupsAlways on: 730 h × DBU per hour × rateAn always-on large warehouse used for tiny lookupsPursue with testing
W3 BackfillC2 Standard mode, or C1 on spotOne large run. Compare both for the volume.C3 all-purpose, or a warehouse running heavy SQL (see next table)Pursue C2 or C1

Table 4

What goes wrong in the mapping

MistakeWhat happensExample from above
Comparing DBU rates onlyClassic rates look low, but VMs and boot time sit on a second bill. Serverless looks high but includes machines.C5 at $0.22 looks three times cheaper than C7 at $0.70, but you still pay cloud VMs for every hour
Scheduled work on all-purpose computeAbout 2.2 times the DBU part for the same jobW1: $445 against $205 a month
Long auto-stop on pro or classic warehousesIdle tails bill DBUs and VMsW4: $980 in DBUs, plus VMs, against $708
Dashboard refreshes keep a warehouse awakeBusy hours multiplyW5: $1,478 against $3,696
Heavy ETL SQL run on a SQL warehouseThe warehouse rate is higher than the jobs rate for the same DBUs. A SQL task in a job can use a warehouse, so check which one it points at.600 DBU: about $420 on a serverless warehouse against $210 on serverless jobs, if DBU use is equal
Streaming on serverlessContinuous and processing-time triggers are not supported, so the design fails or is rebuiltW8
Autoscaling on a streaming clusterDatabricks advises against it, because classic autoscaling is slow to shrink. You pay for the large size.W8
Many tiny jobs on classic clustersYou pay boot time for each, and a small job can bill more for starting than for runningW9 every 15 minutes
Assuming serverless uses the same DBUs as classicIt may use more or fewer. Only your own DBU per run shows the truth.My examples assume equal, which is a simplification
Ignoring Photon's rateA higher DBU rate for no gain on Python-heavy workTest on your real job
Using list prices as your billDiscounts and commitments change the rate, and the system tables show list priceJoin to your contract rate
No tags or budget policyYou cannot tell which workload caused a cost, so the mapping cannot be checkedAll rows
One compute for several workloadsBI, ETL and ad-hoc work share one bill and slow each other downOne warehouse per team and per kind of work
Not checking networking firstServerless may not reach a private resource without extra setup, so the cheap option is not availableSee the networking row on choosing compute

A check you can run each month

  1. For each of your top 10 costs, write down which row of Table 1 it is.
  2. Write down which row of Table 2 it runs on.
  3. If the pair does not match the verdict in Table 3, price the right compute using your own DBU per run and your own rates.
  4. Switch the largest mismatch first, and measure the next month.

Worked decision

Nightly ETL, nobody waiting: serverless Standard mode or a Photon job cluster?

Default to serverless jobs in Standard mode. Use a job cluster only for very large steady work where spot pricing wins, Scala or RDD code, custom images, or private-network access.

Why. Nobody waits, so Standard mode's slow start (typically 4 to 6 minutes, and not billed) costs nothing. It uses fewer DBUs, and Databricks cites savings of up to 70 percent, though the real saving depends on the job. There is no cluster to size, no boot time billed and no VM bill.

Option (60 hours a month, 600 DBU, 5 machines)CalculationMonthly
Job cluster, no Photon600 × $0.15 = $90, plus VMs $115.20$205
Job cluster, spot VMs at a 50 percent discount (assumed)$90 plus $57.60$148
Serverless jobs, default mode600 × $0.35$210
Serverless jobs, Standard, 30 percent fewer DBUs (assumed)420 × $0.35$147
Serverless jobs, Standard, 70 percent fewer DBUs (Databricks' best case)180 × $0.35$63

When does Photon on a job cluster pay off?

Photon raises the DBU rate, so it pays only if the job gets faster by more than the extra cost. Suppose, as an assumption (I did not confirm the multiplier), that Photon doubles the DBU part. The cluster then costs $4.92 an hour (10 DBU × 2 × $0.15 + $1.92 in VMs) instead of $3.42. It breaks even when the job runs about 1.44 times faster. A SQL-heavy job often clears that bar. A Python-heavy job often does not. Serverless already includes Photon, so this question only arises on the classic side.

On paper, serverless Standard and a classic cluster on spot come out close. The deciding factors are effort and boot time. Run both for a week and compare DBUs per run and the total including VMs. The serverless DBU figure is an assumption (equal to classic), and all rates are illustrative.

Checked

What was checked, and what was not

Not confirmed: the official current rates, serverless pipeline rates, DBU ratings per machine type, the Photon multiplier, whether serverless notebooks offer GPU compute, and how serverless DBU use compares with classic for a given job. Replace every dollar figure with your own before you make a decision.

FAQ

Questions people ask

Why can the same job cost so differently on different compute?

Because the price per DBU depends on the product, and classic compute adds a separate cloud bill for machines. In the examples, the same nightly job costs about 2.2 times more on an all-purpose cluster than on jobs compute.

Is serverless more expensive per DBU?

Yes on paper, because the rate includes the machines. Compare the full monthly cost, including cloud VMs, boot time and idle time on classic.

Which numbers on this page can I rely on?

Use them for ratios and mistakes only. The rates are third-party summaries of AWS list prices, not the official price list, and serverless DBU use is assumed equal to classic.

What is the quickest way to find a mismatch?

List your top 10 costs, name the workload and the compute for each, and check them against the mapping table.

Next

Where to go next

Related on this site: Biggest cost and performance levers · Databricks cost calculator · Tagging best practices · Cost and FinOps for the platform

All Databricks compute and cost pages · All data engineering · Databricks on this site