Home › AWS › AWS data services › EKS

AWS data services · Processing and compute · about 10 min read

Amazon Elastic Kubernetes Service (EKS)

Managed Kubernetes on AWS: AWS runs the control plane, and with Auto Mode the worker machines too, while you run containers on it.

RoleProcessing and compute
StatusGenerally available. EKS Auto Mode and EKS Capabilities (managed Argo CD, ACK and kro, launched Dec 2025) are available. Kubernetes 1.34 to 1.37 are in standard support, 1.31 to 1.33 in paid extended support.
Verdict on this siteKeep if you have it
AzureAzure Kubernetes Service (AKS)
Google CloudGoogle Kubernetes Engine (GKE)
Databricksnone: Databricks runs its own managed compute
Open sourceKubernetes, self-managed
Verdict: Keep if you have it. Spark on Kubernetes is a second processing platform to build and run, and Databricks already does the processing. Use EKS only if the company already runs it well, for example to host Kafka Connect workers with Strimzi or the app services that read gold data.

In plain words

What it is

Kubernetes is an open-source system for running many containers across many machines. You describe what should run, for example three copies of an API and one copy of a worker, in files called manifests, and Kubernetes keeps reality matching that description: it places containers, called pods, onto machines, called nodes, restarts them when they fail, and scales them up and down. It is powerful and portable, and it has a lot of moving parts.

Amazon Elastic Kubernetes Service, or EKS, is Kubernetes run by AWS. AWS runs the control plane, the brain that stores the desired state and makes decisions, across several availability zones. You choose how the nodes are run: your own EC2 machines, managed node groups, Fargate, or EKS Auto Mode, where AWS also picks, patches and replaces the EC2 nodes, with each node living at most 21 days. Since December 2025, EKS Capabilities can also run Argo CD, AWS Controllers for Kubernetes and kro for you, outside your nodes.

A manufacturing example: a company's digital team already runs its dealer portal and warranty-claim APIs on EKS, with a platform team, GitOps deployments and on-call rotations. When the data team needs to move events from the plants' on-premises Kafka brokers into AWS, they deploy Kafka Connect workers on the same cluster with the Strimzi operator, write the events onto Kinesis or MSK, and let Databricks take them from there. The data team reuses a platform that already exists rather than building one.

A large automated warehouse with its own robots and rules: excellent when you already run one, a big project to build just to store a few boxes.

Big picture

Where it fits

SourcesMove inStreamStoreProcessServeUseAcross everything: catalog · security · orchestration · observability and cost · infrastructure as codegets data fromhands data toOn-premises Kafka and plant systemsAmazon MSKApps and partner APIsEKSProcessing and computeKinesis Data StreamsS3Apps that read gold tables

The lit lane is where EKS works. Boxes on the left feed it; boxes on the right use what it produces.

The reference design does not need EKS. Processing, orchestration and governance live in Databricks, ingestion uses managed movers such as DMS, AppFlow, Transfer Family and Kinesis, and the fast lane uses Managed Flink and DynamoDB. Adding a Kubernetes cluster adds upgrades every few months, node patching, add-ons, networking and security policies, and the skills to run them.

Where EKS earns a place is at the edges, when it already exists. In the Move in lane it can host long-running connectors, such as Kafka Connect or Debezium workers, that land data in S3, Kinesis or MSK. In the Serve lane it can host the company's own APIs and apps that read gold tables through a Databricks SQL warehouse or Lakebase. In both cases the data work after landing, and the business logic over the tables, stays in Databricks.

Spark on EKS, through the Spark operator or EMR on EKS, is the case to resist. It looks cheap because the cluster is already there, but you take on Spark images, autoscaling, shuffle storage, a catalog, lineage and access control yourself, and the data now lives under two engines with two permission models. The site's verdict on EMR and Glue jobs applies here too: new processing runs on Databricks.

How it works

The ideas you need

IdeaWhat it means
Control planeThe Kubernetes API server and its database, run by AWS across availability zones. You pay a fixed hourly fee per cluster.
Node, pod and deploymentA node is a machine; a pod is one or more containers that run together; a deployment keeps a set number of identical pods running.
Managed node groups and KarpenterTwo ways to supply EC2 nodes. Managed node groups are Auto Scaling groups EKS helps manage; Karpenter starts the right instance type for waiting pods.
EKS Auto ModeAWS manages the nodes, autoscaling, load balancers, storage drivers and pod networking. Nodes use locked-down Bottlerocket images and are replaced at least every 21 days.
Standard and extended supportEach Kubernetes minor version gets 14 months of standard support, then 12 months of paid extended support, after which AWS upgrades the control plane for you.
OperatorA program running in the cluster that manages a complex application, such as Strimzi for Kafka or the Spark operator for Spark jobs.
EKS Pod Identity and IRSAWays to give a pod its own IAM role, so a Kafka Connect worker can write to one stream without the whole node having that permission.
EKS CapabilitiesManaged Argo CD (GitOps deployments), AWS Controllers for Kubernetes (create AWS resources from manifests) and kro (group resources), run by AWS outside your nodes and billed per hour.
EMR on EKSAn option to run Amazon EMR's Spark on an existing EKS cluster, sharing nodes with other workloads.

Choosing

When to use it, and when not

Use it when

  • The company already runs EKS with a platform team, upgrade routine, GitOps and on-call, and offers it to the data team as a service.
  • You need long-running connectors, such as Kafka Connect, Debezium or MQTT bridges, alongside other Kubernetes workloads.
  • The apps that read gold data, such as a dealer portal or a store replenishment API, already live on Kubernetes.
  • You must run the same container platform on several clouds or on premises, and Kubernetes is the agreed common layer.
  • Workloads need operators or Kubernetes-native tools that have no simpler managed equivalent on AWS.

Avoid it when

  • The only reason is to run Spark or Airflow more cheaply than Databricks or MWAA; you would be building and staffing a second platform.
  • The data team would be the only user and nobody has run Kubernetes upgrades in production.
  • A few containers are all you need; ECS on Fargate does that with far less to operate.
  • A managed mover such as DMS, AppFlow, Firehose or MSK Connect already covers the source.
  • You cannot commit to upgrading Kubernetes roughly every year, which is what avoiding extended support fees requires.

Production grade

What goes wrong in production

ProblemWhat breaksWhat to do
Version treadmillA minor version leaves standard support after 14 months; the cluster moves to extended support at six times the control plane price, then is upgraded automatically.Plan an upgrade at least once a year, test add-ons and operators on a staging cluster, and alarm on the end-of-support dates.
Add-on and API driftAn upgrade removes an old Kubernetes API that a Helm chart or operator still uses, and deployments fail.Scan manifests for deprecated APIs before each upgrade and keep operators such as Strimzi current.
IP address exhaustionEach pod takes a VPC IP address; busy clusters run out of addresses in small subnets and pods stay pending.Size subnets generously, use prefix delegation or a secondary CIDR range, and watch available IPs per subnet.
Connector restarts and lagKafka Connect workers rebalance on every pod restart, and node replacement every few weeks causes lag spikes.Set pod disruption budgets, spread workers across zones, and alarm on connector lag rather than pod health.
Overly broad permissionsA node role with write access to every bucket lets any pod on the node write anywhere.Use EKS Pod Identity per service account, keep node roles minimal, and audit with CloudTrail.
Spark on Kubernetes sprawlA team runs Spark jobs on the cluster; tables now have two engines, two permission models and no shared lineage.Keep processing on Databricks; allow only movers and apps on EKS, enforced by namespace policy.
Hidden costsLoad balancers, NAT traffic, logs and idle nodes cost more than the control plane fee.Tag namespaces, use Auto Mode or Karpenter consolidation, and review the bill per namespace monthly.

Alternatives

What to compare it with

AlternativePick the alternative whenPick EKS when
Amazon ECS and AWS FargateYou need a few containers and no Kubernetes skills or ecosystem.The company runs Kubernetes already and wants one platform for all containers.
Amazon MSK ConnectYou only need managed Kafka Connect workers and no other containers.Connectors need custom plugins, operators or sit next to other services on the cluster.
Databricks jobs and appsThe workload is processing, quality checks or a data app over Unity Catalog tables.The workload is an always-on connector or a company app that already lives on Kubernetes.
Amazon EMR on EKSYou must keep EMR Spark jobs running and want them on shared nodes during a migration.You are not running Spark at all on the cluster.
Azure AKS or Google GKEThe platform runs on Azure or Google Cloud; the role is the same.You are on AWS.

Cost

How you pay

Each cluster pays an hourly control plane fee: list price 0.10 US dollars per hour in standard support and 0.60 per hour in extended support, check your region. Nodes are billed as EC2 instances, Fargate pods or Auto Mode instances; Auto Mode adds a per-instance management fee on top of the EC2 price. EKS Capabilities bill a base rate per hour plus a usage rate per managed application or resource. An optional Provisioned Control Plane adds larger fixed hourly tiers. Load balancers, EBS, NAT and logs are billed by their own services.

What drives the bill

  • The EC2 nodes, by far the biggest line; utilisation decides how much of them you waste.
  • Number of clusters, each with its own control plane fee and add-ons.
  • Extended support fees for clusters that are not upgraded in time.
  • Load balancers, NAT gateway traffic and cross-zone data transfer.
  • CloudWatch Container Insights and log volume.

Cost traps

  • A cluster per team or per environment multiplies fixed fees, add-ons and upgrade work.
  • Extended support starts automatically when standard support ends; a forgotten dev cluster pays six times the control plane price.
  • Pods that request far more CPU and memory than they use keep large nodes alive.
  • Counting only the AWS bill: people time for upgrades and incidents is the largest cost of running Kubernetes.

Prices change; check the AWS pricing page for your region before deciding.

Security and governance

Keeping it safe

  • Keep the cluster API endpoint private, or restrict public access to known address ranges, and run nodes in private subnets without public IPs.
  • Give each workload its own IAM role through EKS Pod Identity, scoped to one stream, one bucket prefix or one secret; keep the node role minimal.
  • Map people to Kubernetes permissions with EKS access entries and your identity provider, and keep cluster-admin to a small group.
  • Encrypt Kubernetes secrets with a KMS key, or better, read secrets from Secrets Manager at run time.
  • Use network policies so connector pods can reach only their sources and AWS endpoints; reach S3, STS, ECR and Kinesis through VPC endpoints.
  • Send control plane audit logs to CloudWatch and keep CloudTrail on; do not let pods log the personal data that passes through connectors.

Operating it

What to watch, and the limits

WatchWhy
Pods pending and node countPods that cannot be placed show capacity or IP shortages before users notice.
Pod restarts and OOMKilled eventsCrash loops or memory kills in connectors and apps.
Connector lagFor Kafka Connect or Debezium, how far behind the source the workers are; the real health signal for data.
Control plane version and end-of-support dateWhen the cluster will move to paid extended support.
Available IPs per subnetRunning out stops new pods from starting.
Cost per namespaceWhich workloads drive the node bill, from tags and split cost allocation data.
LimitValue and notes
Standard support14 months per Kubernetes minor version
Extended supportA further 12 months at a higher hourly price, then automatic control plane upgrade
Auto Mode node lifetimeAt most 21 days, then the node is replaced
EKS CapabilitiesOne of each capability type (Argo CD, ACK, kro) per cluster
Pod IP addressesEach pod takes a VPC IP address by default, so subnet size limits pod count

Limits and quotas checked 2026-11-11; many can be raised through Service Quotas.

Decisions

Situations and verdicts

SituationChoiceVerdict
Company already runs EKS; plant Kafka must reach AWSKafka Connect with Strimzi on the shared cluster, writing to MSK or KinesisKeep if you have it: reuse the platform and its people; keep the workers to moving data.
No Kubernetes in the company; one custom API puller neededECS on Fargate or a Databricks job instead of a new EKS clusterDrop: a Kubernetes cluster for one container is all cost and no gain.
Team proposes Spark on EKS to cut Databricks spendSpark operator or EMR on EKS for silver and goldDrop: a second processing platform with no shared catalog or lineage.
Store replenishment API reading gold tablesRun the API on the existing EKS cluster, reading through a Databricks SQL warehouse or LakebasePursue with care: fine where the app team owns EKS; keep the logic over tables in Databricks.

FAQ

Questions people ask

Do I need Kubernetes for a Databricks data platform on AWS?

No. Databricks runs its own compute, and AWS managed movers cover most sources. EKS only makes sense if the company already runs it and offers it as a shared service.

What is EKS Auto Mode?

An EKS option where AWS manages the worker nodes, autoscaling, load balancing, storage and pod networking as well as the control plane. Nodes are replaced at least every 21 days and you pay a management fee on top of EC2.

How long is a Kubernetes version supported on EKS?

Fourteen months of standard support from its EKS release, then twelve months of extended support at a higher price. After that AWS upgrades the control plane automatically, and your nodes and add-ons still need updating.

Should I run Spark on EKS instead of Databricks?

Only if you are ready to build and staff a Spark platform: images, autoscaling, shuffle, catalog, lineage and security. For most data teams the saving disappears in engineering time, and data ends up governed in two places.

ECS or EKS for a few data containers?

ECS on Fargate. It has no control plane fee, no version upgrades and far fewer concepts to learn.

Sources

Checked against

Checked 2026-11-11.

Related

Where it connects

Data platform on AWS and Databricks · Streaming on AWS and Databricks · Amazon ECS and AWS Fargate · Amazon EMR · Amazon MSK · All AWS data services

← ECS and Fargate · Glue ETL jobs →