Home › AWS › AWS data services › PrivateLink
AWS data services · Security, identity and network · about 11 min read
AWS PrivateLink (incl. Databricks front-end and back-end PrivateLink)
Private, one-way connections to a service through an address inside your own network, so traffic never crosses the internet.
In plain words
What it is
AWS PrivateLink lets one network use a service that lives in another network without the two networks being joined and without any traffic touching the internet. The provider puts its service behind a Network Load Balancer and publishes it as an endpoint service. The consumer creates an interface VPC endpoint: a network card with a private IP address inside the consumer's own subnet. Anything sent to that address arrives at the provider's service over the AWS network. Connections can only be opened from the consumer side, so the provider cannot wander into the consumer's network.
Most AWS services, such as STS, Kinesis and Secrets Manager, are offered this way, and so are many vendors, including Databricks. A company can also publish its own service, for example an internal API in one account that other accounts consume privately.
Take a retailer whose analysts open Databricks dashboards on sales and stock from the head office, while nightly clusters build those tables from SAP orders. The security team wants proof that neither the analysts' browsers nor the clusters ever talk to Databricks over the public internet. PrivateLink gives both paths a private address: the analysts' laptops reach the workspace through an endpoint in a network linked to the office, and the clusters reach the Databricks control plane through endpoints in their own VPC. If the public route is then blocked in the workspace settings, a stolen password used from a café gets nowhere.
A private pneumatic tube from your office straight into the supplier's mailroom: only you can send through it, and it never passes through the street.
Big picture
Where it fits
The lit lane is where PrivateLink works. Boxes on the left feed it; boxes on the right use what it produces.
Databricks on AWS has two halves. The control plane, run by Databricks, hosts the web application, the REST APIs, the job scheduler and Unity Catalog. The compute plane is where data is processed: classic clusters run in your VPC, and serverless compute runs in Databricks' own network in the same region. PrivateLink connects these halves in three places, and the reference platform in Mumbai uses all three.
Back-end PrivateLink connects classic clusters to the control plane. You create two interface endpoints in the workspace VPC, one for the workspace REST APIs and one for the secure cluster connectivity relay (the channel clusters use to receive work, so that clusters need no public IP and no open inbound ports). Front-end PrivateLink, which Databricks now calls inbound PrivateLink, connects people and tools to the workspace. Its endpoint usually sits in a separate transit VPC that the office network reaches over Direct Connect or VPN through the Transit Gateway. Serverless private endpoints run the other way: a network connectivity configuration (NCC) in the Databricks account holds private endpoint rules that let serverless SQL, jobs and pipelines reach services you publish in your VPC, such as a database behind a Network Load Balancer.
PrivateLink sits beside the plain AWS service endpoints (S3, STS, Kinesis) in the Terraform layer 2-network, and is the reason the reference platform can say there is no public IP and no internet path anywhere. CloudTrail, VPC Flow Logs and the Databricks audit system tables together prove it to an auditor.
How it works
The ideas you need
| Idea | What it means |
|---|---|
| Endpoint service | A service a provider publishes for PrivateLink, sitting behind a Network Load Balancer. |
| Interface endpoint | The consumer's private network card that connects to an endpoint service; billed per hour per zone and per gigabyte. |
| Back-end PrivateLink | Private connection from classic clusters in your VPC to the Databricks control plane, using a workspace endpoint and a relay endpoint. |
| Front-end (inbound) PrivateLink | Private connection from users, BI tools and APIs to the workspace web application and REST APIs. |
| Secure cluster connectivity relay | The outbound-only channel classic clusters use to take instructions from the control plane, so they need no public IP. |
| VPC endpoint registration | Telling the Databricks account about an AWS endpoint ID so it will accept traffic from it; a registration cannot be edited later. |
| Private access settings | A Databricks account object that says whether a workspace still accepts public traffic and which registered endpoints may reach it. |
| Network connectivity configuration (NCC) | A Databricks account object holding the private endpoint rules and stable addresses that serverless compute uses to reach your resources. |
| Transit VPC | A small VPC that holds the front-end endpoint and is joined to the office network, kept apart from the compute VPC. |
Choosing
When to use it, and when not
Use it when
- Use back-end PrivateLink whenever classic Databricks clusters run in your VPC and the security team requires that cluster-to-control-plane traffic never touches the internet.
- Use front-end PrivateLink when users and BI tools sit on a corporate network and the workspace must refuse sign-ins from anywhere else.
- Use NCC private endpoint rules when serverless jobs or SQL must reach a private database, an internal API or a Kafka cluster in your VPC.
- Use interface endpoints for AWS services such as STS, Kinesis and Secrets Manager when compute has no internet route at all.
- Publish your own endpoint service when one account offers an internal API or dataset to other accounts without joining their networks.
Avoid it when
- Avoid PrivateLink to Databricks when the account is not on the Enterprise tier; it is not offered below that.
- Avoid building front-end PrivateLink before the network and DNS teams have agreed how office DNS will resolve the workspace URL; a wrong DNS setup locks everyone out.
- Do not use an S3 interface endpoint through PrivateLink for traffic that starts inside the VPC; the S3 gateway endpoint is free.
- Avoid publishing a database through an NCC private endpoint when it changes IP addresses on failover and nothing keeps the load balancer targets in step.
Production grade
What goes wrong in production
| Problem | What breaks | What to do |
|---|---|---|
| Settings that cannot be changed | Endpoint registrations and the workspace network configuration cannot be edited once created; a mistake means building new objects and in some cases a new workspace. | Write the whole design in Terraform first, review it, build it in dev, and only then apply the same code to pre-prod and prod. |
| DNS does not resolve to the private address | The workspace URL still resolves to a public address, so traffic goes over the internet or fails when public access is blocked. | Turn on private DNS on the endpoints, point the workspace URL at the endpoint in office DNS, and check with nslookup that the answer is a privatelink name with a private IP. |
| Users locked out after blocking public access | Public access is blocked before every user path and identity-provider redirect has been tested, and nobody can sign in. | Start with public access governed by ingress policies, test from every office and tool, then switch to block; keep an account admin break-glass path. |
| Browser sign-in still needs the internet | Single sign-on redirects to the Databricks account sign-in page, which by default is a public address. | If users truly have no internet, set up the private browser authentication path with a private hosted zone; it is in Beta, so test it carefully. |
| Clusters fail to start after workspace creation | Clusters launched in the first minutes after a PrivateLink workspace shows running fail to connect. | Wait about 20 minutes after the workspace is running before creating clusters, as Databricks documents, and retry rather than redesign. |
| Security group ports missing | The relay endpoint needs port 6666 and the workspace endpoint port 443, plus 2443 with the compliance security profile; one missing rule leaves clusters waiting forever. | Use separate security groups for clusters and endpoints, take the port list from the Databricks customer-managed VPC page, and check VPC Flow Logs for rejects. |
| Serverless private endpoint stuck pending | The NCC rule never becomes established because the endpoint service does not allow the Databricks principal or the request was not accepted. | Allow the Databricks serverless role on the endpoint service, accept the connection in AWS, and check the rule shows ESTABLISHED before attaching the NCC to workspaces. |
| NCC quotas reached | A large organisation with many workspaces and private databases runs into the per-region NCC and private endpoint limits. | Plan one NCC per environment, share endpoint services where you can, and ask Databricks early if you need more. |
Alternatives
What to compare it with
| Alternative | Pick the alternative when | Pick PrivateLink when |
|---|---|---|
| VPC peering or Transit Gateway | You control both networks and need many services or two-way traffic between them. | The other side belongs to a vendor; PrivateLink gives a one-way, single-service connection without merging address ranges. |
| Public endpoint with IP access lists | A small team, no regulator asking, and you can list fixed office addresses. | Security requires that traffic never cross the internet or that stolen credentials be useless outside the network. |
| Databricks serverless without private endpoints | Serverless only reads your S3 buckets and public SaaS, which works without extra network setup. | Serverless must reach a private database or API in your VPC. |
| Azure Private Link | The platform runs on Azure Databricks; the same front-end and back-end ideas apply. | You are on AWS. |
| Google Private Service Connect | The platform runs on Databricks on Google Cloud. | You are on AWS. |
| Site-to-Site VPN straight into the compute VPC | A plant or office must reach many private systems in AWS. | The goal is only to reach Databricks privately; a front-end endpoint in a transit VPC is narrower and easier to audit. |
Cost
How you pay
On AWS, each interface endpoint is billed for every hour it exists in each availability zone, plus a charge per gigabyte processed that falls in tiers as monthly volume grows (about 0.01 USD per endpoint per zone per hour and 0.01 USD per GB for the first tier in us-east-1, list price, check your region). Providers who publish endpoint services pay for their Network Load Balancer. Databricks does not charge for classic PrivateLink endpoints itself, but serverless private endpoints and their data processing show up as networking charges on the Databricks bill.
What drives the bill
- The number of interface endpoints times zones times environments, including the workspace, relay, front-end, STS, Kinesis and Secrets Manager endpoints.
- Gigabytes through endpoints: for Databricks mostly result sets and notebook traffic, for AWS services mostly Kinesis.
- Network Load Balancers you run for your own endpoint services used by serverless.
- Databricks serverless networking charges for private connectivity to your resources.
Cost traps
- A front-end endpoint per environment and per transit VPC multiplies hourly charges; one front-end endpoint can be registered with several workspaces and accounts.
- Large query results returned to BI tools through front-end PrivateLink are charged per gigabyte; extracts that pull whole tables add up.
- Network Load Balancers left behind after a serverless test keep billing every hour.
- Serverless networking costs land on the Databricks invoice, so an AWS-only cost review misses them.
Prices change; check the AWS pricing page for your region before deciding.
Security and governance
Keeping it safe
- Block public access in the workspace's private access settings once testing is done, so the workspace accepts only registered endpoints.
- Use the private access level that lists explicit endpoints where possible, so a new endpoint registered elsewhere in the account cannot reach the workspace by default.
- Keep the front-end endpoint in a transit VPC joined to the office network, separate from the compute VPC that holds data traffic.
- Require acceptance on your own endpoint services and allow only the Databricks serverless principal, so no other account can attach to them.
- Log endpoint traffic with VPC Flow Logs and keep CloudTrail records of endpoint creation and acceptance in the log-archive account.
- PrivateLink hides the path, not the identity: keep single sign-on, service principals and Unity Catalog grants as the real access control.
Operating it
What to watch, and the limits
| Watch | Why |
|---|---|
| BytesProcessed (AWS/PrivateLinkEndpoints) | Traffic through each endpoint, which follows the per-gigabyte charge. |
| NewConnections and ActiveConnections (AWS/PrivateLinkEndpoints) | Whether clusters and users are actually using the endpoint; zero after go-live means traffic is taking another path. |
| PacketsDropped (AWS/PrivateLinkEndpoints) | Packets the endpoint dropped, often from security group or size issues. |
| NCC private endpoint rule state | Should read ESTABLISHED; PENDING or REJECTED means serverless cannot reach the resource. |
| VPC Flow Logs REJECT records on endpoint network cards | Ports or security groups blocking the relay or workspace endpoint. |
| Databricks audit logs sign-in source | Confirms sign-ins arrive through the private endpoint and not from public addresses. |
| Limit | Value and notes |
|---|---|
| Interface and Gateway Load Balancer endpoints per VPC | 50 by default, adjustable |
| Endpoint bandwidth | 10 Gbps per zone by default, scaling to 100 Gbps |
| Packet size through an endpoint | MTU of 8,500 bytes; larger packets are dropped |
| Databricks NCCs | Up to 20 per account per region, each attached to up to 50 workspaces |
| Databricks serverless private endpoints | Up to 30 per region across NCCs, with up to 100 domain names per rule |
Limits and quotas checked 2026-11-11; many can be raised through Service Quotas.
Decisions
Situations and verdicts
| Situation | Choice | Verdict |
|---|---|---|
| Classic clusters in prod must never reach Databricks over the internet | Back-end PrivateLink with workspace and relay endpoints in both zones | Pursue: required for the no-internet design, and cheap compared with the compute it protects. |
| Analysts and BI tools in the head office | Front-end PrivateLink in a transit VPC reached over Direct Connect, public access blocked after testing | Pursue: a stolen password is useless outside the corporate network. |
| Serverless pipeline must read a private plant database | NCC private endpoint rule to a Network Load Balancer in front of the database | Pursue with care: works well, but keep load balancer targets in step with database failover. |
| Small dev workspace used by three engineers | Public access with IP access lists and single sign-on | Pursue with care: acceptable for dev without real data; never for prod. |
| Account on a tier below Enterprise | Upgrade to Enterprise before go-live | Pursue: PrivateLink and private access settings need it, and retrofitting the network later is painful. |
FAQ
Questions people ask
What is the difference between front-end and back-end PrivateLink in Databricks?
Back-end PrivateLink connects classic clusters in your VPC to the Databricks control plane. Front-end, also called inbound, PrivateLink connects users, BI tools and API callers to the workspace. You can use either or both; the reference platform uses both.
Do I need a customer-managed VPC for Databricks PrivateLink?
Yes for classic compute, on both front-end and back-end. A serverless-only workspace does not need one, because its compute runs in the Databricks network.
Does serverless compute use PrivateLink?
Serverless reaches your private resources through private endpoint rules in a network connectivity configuration, which use PrivateLink to an endpoint service you publish behind a Network Load Balancer.
Does PrivateLink replace authentication?
No. It only makes the path private. Users still sign in through your identity provider, jobs run as service principals, and Unity Catalog grants decide what each one can read.
Which Databricks plan do I need?
The Enterprise tier. Check this before designing the network, because PrivateLink and private access settings are not available below it.
Sources
Checked against
- Databricks docs: Classic compute plane PrivateLink
- Databricks docs: Inbound PrivateLink for workspaces
- Databricks docs: Serverless private connectivity to resources in your VPC
- Databricks docs: Customer-managed VPC
- AWS docs: AWS PrivateLink quotas
- AWS PrivateLink pricing
Checked 2026-11-11.
Related
Where it connects
AWS and Databricks reference platform · VPC, endpoints and NAT · Direct Connect, VPN and Transit Gateway · AWS IAM and STS · AWS on this site · All AWS data services