Home › AWS › AWS data services › S3 Tables

AWS data services · Storage and table formats · about 11 min read

Amazon S3 Tables

A special S3 bucket type that stores Apache Iceberg tables, holds their catalog pointer and runs compaction and clean-up for you.

RoleStorage and table formats
StatusGenerally available. Supports Apache Iceberg v3, Intelligent-Tiering for tables and table replication (checked 2026-11-11)
Verdict on this siteKeep if you have it
AzureNo direct twin; Iceberg or Delta on ADLS with Fabric OneLake or Unity Catalog managing it
Google CloudBigLake Iceberg tables (BigLake metastore)
DatabricksUnity Catalog managed tables (Delta or managed Iceberg) with Predictive Optimization
Open sourceApache Iceberg with a REST catalog such as Apache Polaris or Lakekeeper, plus scheduled maintenance jobs
Verdict: Keep if you have it. On an AWS-only platform it is the AWS twin of a managed table and worth pursuing. On this Databricks platform, Unity Catalog managed tables already give you open files in your bucket with Predictive Optimization doing the chores. Read S3 Tables that another team already owns in place through catalog federation, and never make AWS the owner of tables Databricks writes.

In plain words

What it is

A table in a data lake is not one file. It is many Parquet data files plus a set of metadata files that say which data files make up the current version of the table. Apache Iceberg is one open format for that metadata. Someone has to keep a pointer to the newest metadata file (that is the job of a catalog), and someone has to tidy up: merge small files into bigger ones, drop old versions and delete files nothing points to any more. On plain S3 you assemble all of that yourself.

Amazon S3 Tables packages those three jobs into one AWS service. You create a table bucket, which is a different kind of bucket from the general purpose buckets most people know. Inside it you create namespaces (folders for tables, like schemas in a database) and tables. Every table is an Iceberg table. S3 keeps the catalog pointer, and it runs compaction, snapshot management and unreferenced file removal in the background, switched on by default.

Picture a retailer with no Databricks. Orders from the web shop arrive through Amazon Data Firehose every few minutes and land as a table called sales.orders in a table bucket. Analysts query it from Athena, a nightly Glue job adds returns, and Redshift reads it for finance reports. Nobody writes a compaction job: the thousands of small files Firehose leaves behind are merged in the background, and versions older than the retention window disappear on their own.

Because the tables live inside S3, they keep S3 durability and scale, but they are reached through their own API (the s3tables namespace), through the AWS Glue Data Catalog integration, or through an Iceberg REST endpoint that open-source engines can call.

A filing cabinet that comes with its own index card box and a night cleaner who staples loose pages together and shreds drafts nobody needs.

Big picture

Where it fits

SourcesMove inStreamStoreProcessServeUseAcross everything: catalog · security · orchestration · observability and cost · infrastructure as codegets data fromhands data toData FirehoseGlue ETL jobsAmazon EMRS3 TablesStorage and table formatsAthenaRedshiftDatabricks through catalog federation (read)

The lit lane is where S3 Tables works. Boxes on the left feed it; boxes on the right use what it produces.

S3 Tables sits in the store lane, next to general purpose S3 buckets. On the site's reference platform, the medallion tables (bronze, silver, gold) are Unity Catalog managed tables in ordinary S3 buckets, one bucket and one KMS key per catalog. Databricks owns them and Predictive Optimization does the chores. S3 Tables is not part of that main path.

It shows up in two situations. First, an AWS-only team, or an older part of the estate, already keeps Iceberg tables in a table bucket, fed by Firehose, Glue or EMR and read by Athena or Redshift. Databricks then reads those tables in place through Unity Catalog catalog federation, as foreign tables another catalog owns. Copying them is the wrong answer, because it doubles storage and creates two truths.

Second, a team without Databricks needs a lakehouse of its own. There S3 Tables is a good default, because it gives the AWS engines a managed table with maintenance built in. The site's table maintenance page puts it simply: whoever holds the pointer owns the table, and the owner holds the broom. Pick one owner per table and let every other engine be a guest.

How it works

The ideas you need

IdeaWhat it means
Table bucketA bucket type made only for tables. Block Public Access is always on and cannot be turned off. It is separate from general purpose and directory buckets.
NamespaceA logical folder inside a table bucket that groups tables, much like a schema in a database.
Apache IcebergThe open table format every S3 table uses. Data stays in Parquet files; Iceberg metadata files describe each version of the table.
SnapshotOne version of an Iceberg table. Every write makes a new snapshot; old snapshots allow time travel until they are expired.
CompactionRewriting many small data files into fewer large ones so queries open fewer files. S3 Tables runs it automatically per table.
Unreferenced file removalDeleting files that no snapshot points to, such as leftovers from failed writes. Set at the table bucket level.
Glue Data Catalog integrationA switch that publishes table buckets into the Glue Data Catalog so Athena, Redshift, EMR and Quick can find the tables, with Lake Formation for permissions.
Iceberg REST endpointA standard Iceberg catalog API that S3 Tables exposes, so Spark, Trino, PyIceberg and others can reach tables without AWS-specific plugins.
Catalog federationHow Unity Catalog shows tables owned by another catalog, such as Glue or S3 Tables, as foreign tables it can read and govern without copying.

Choosing

When to use it, and when not

Use it when

  • You run an AWS-only lakehouse with Athena, Glue, EMR and Redshift, and want Iceberg tables whose upkeep AWS runs for you.
  • Firehose or a streaming job writes many small files into an Iceberg table and nobody wants to own a compaction schedule.
  • Another team already owns Iceberg tables in a table bucket and your Databricks platform only needs to read them through catalog federation.
  • You want access rules per table or per namespace in IAM, using the separate s3tables actions, instead of prefix rules in a bucket policy.
  • A small analytics team without a platform team needs managed tables quickly and is happy to stay inside AWS engines.

Avoid it when

  • Databricks is your main engine. Unity Catalog managed tables already keep open files in your bucket, and Predictive Optimization, row filters, column masks and lineage need tables Unity Catalog owns.
  • You want Delta Lake. S3 Tables stores only Iceberg, so Delta pipelines, change data feed and Lakeflow streaming tables do not fit.
  • You plan to write the same table from Databricks and from AWS engines. Two writers through two catalogs means two permission systems and clean-up jobs that can delete files the other still needs.
  • You need every file to sit at a path you choose in a general purpose bucket, for example for an existing replication or Object Lock setup built on prefixes.

Production grade

What goes wrong in production

ProblemWhat breaksWhat to do
Two owners for one tableDatabricks writes a table through the open-source Iceberg library while S3 Tables also maintains it. Unity Catalog governance is bypassed, and lineage and audit miss half the writes.Give every table one owner. On this platform, Databricks-written tables are Unity Catalog managed tables; S3 Tables are read through federation.
Permissions in two placesLake Formation grants on the Glue side and Unity Catalog grants on the Databricks side drift apart, so a user is blocked in one engine and allowed in the other.Treat federated S3 Tables as read-only in Databricks, review both grant sets together and record who owns each table in the metadata store.
Federation writes are limitedTeams expect Databricks to write into foreign Iceberg tables as it writes its own, then find the feature limited or in preview.Plan for reads only and check the current Databricks catalog federation docs before promising writes.
Retention set by defaultSnapshot management expires old versions on default settings, then an auditor asks for the table as it was 30 days ago.Set snapshot retention per table on purpose, match it to time-travel needs and write the setting down.
Maintenance bill surprisesCompaction is billed per object and per byte processed. A table rewritten every few minutes by a streaming job can cost more in compaction than in storage.Watch maintenance charges in the Cost and Usage Report per table bucket; batch writes into larger commits where you can.
Copying tables for a second engineA nightly job copies Delta gold tables into S3 Tables so Athena can read them: double storage, a lag and two answers.Let Athena read Unity Catalog tables as Iceberg through UniForm and the Iceberg REST catalog instead of copying.
Quota surprises in multi-tenant designsA design with one table bucket per team or per plant hits the default of 100 table buckets per Region.Use namespaces for separation inside fewer buckets, or request a quota increase early.

Alternatives

What to compare it with

AlternativePick the alternative whenPick S3 Tables when
Unity Catalog managed tables (Delta or managed Iceberg)Databricks is the main engine. You want Predictive Optimization, masks, row filters, lineage and one permission model. This is the site's default.No Databricks at all; then S3 Tables is the simpler choice.
Glue Data Catalog Iceberg tables with table optimizersYou want Iceberg in your own general purpose bucket, at paths you choose, with AWS running compaction and retention as Glue compute.You prefer one service that bundles storage, catalog and maintenance with less setup.
Iceberg on plain S3 with your own jobsYou need full control over file layout and maintenance timing, or you run engines that the AWS catalog cannot serve.Nobody has time to own compaction, snapshot expiry and orphan clean-up.
Azure: Iceberg or Delta on ADLS with Fabric OneLakeYou are on Azure; OneLake keeps Delta and serves Iceberg readers.Your platform is on AWS.
Google Cloud: BigLake Iceberg tablesYou are on Google Cloud and want BigQuery to manage Iceberg tables in Cloud Storage.Your platform is on AWS.

Cost

How you pay

You pay for table storage per GB-month, as with S3 but at a table bucket rate; for requests against the tables; for a small monitoring fee per object; and for maintenance, which is billed per object processed and per GB processed by compaction. Intelligent-Tiering can now move table data that is not read to cheaper access tiers. Query engines such as Athena, Redshift or EMR bill separately for reading. Check the S3 pricing page for your Region; rates differ from general purpose buckets.

What drives the bill

  • How much data the tables hold after compaction and snapshot expiry.
  • How often the table is written: many small commits mean more compaction work.
  • How long snapshots are kept, because old versions keep old files alive.
  • Number of objects, which drives the monitoring fee and per-object compaction charges.

Cost traps

  • Streaming writers making a commit every few seconds, so compaction rewrites the same data again and again.
  • Long snapshot retention left on by default for tables that nobody time-travels.
  • Copying tables from Databricks into S3 Tables, which pays for storage twice plus a copy job.
  • Forgetting that replication of tables to another Region bills storage and requests in the destination too.

Prices change; check the AWS pricing page for your region before deciding.

Security and governance

Keeping it safe

  • Access uses its own IAM namespace, s3tables, so policies can grant per table, per namespace or per table bucket, and Service Control Policies can block whole actions across accounts.
  • Block Public Access is always on for table buckets and cannot be switched off.
  • Encrypt with SSE-KMS using a customer managed key per table bucket where the data is sensitive, in line with the site's one key per catalog rule.
  • Reach S3 Tables privately from your VPC with AWS PrivateLink interface endpoints, so traffic never needs a public route.
  • Record management and data access in AWS CloudTrail. When tables are published to the Glue Data Catalog, Lake Formation grants and audit apply on top.
  • Personal data should be tokenised before it reaches any table, as on the rest of the platform; S3 Tables does not mask columns for Databricks readers.

Operating it

What to watch, and the limits

WatchWhy
Maintenance job statusWhether compaction, snapshot management and unreferenced file removal last ran and succeeded, per table, from the S3 Tables maintenance status API or console.
Files per table and average file sizeRead from the Iceberg metadata tables; many small files mean compaction is behind.
Snapshot count and oldest snapshotShows whether retention is doing its job and whether time travel reaches as far back as promised.
CloudTrail s3tables eventsWho created, changed or deleted tables and maintenance settings.
Cost and Usage Report lines for S3 TablesStorage, request, monitoring and compaction charges per table bucket, to spot a runaway table.
Federated reads from DatabricksUnity Catalog audit system tables for queries against the foreign catalog, to see who uses the tables and how often.
LimitValue and notes
Table buckets100 per AWS Region per account by default; increase through Support
Namespaces10,000 per table bucket by default; adjustable
Tables10,000 per table bucket by default; adjustable
Table formatApache Iceberg only, including Iceberg v3 features such as deletion vectors and row lineage; no Delta Lake
Table migrationUp to 1 billion source objects and 100,000 partitions per migration; 10 concurrent migrations per account

Limits and quotas checked 2026-11-11; many can be raised through Service Quotas.

Decisions

Situations and verdicts

SituationChoiceVerdict
Databricks platform writing bronze, silver and gold for plant sensor data and SAP ordersUnity Catalog managed tables in your own S3 buckets, Iceberg reads through UniForm for AthenaDrop S3 Tables here: Databricks must own the tables it writes so Predictive Optimization, masks and lineage work.
A retail analytics team already keeps web orders in a table bucket, and the Databricks platform needs them for a margin reportRead the table in place through Unity Catalog catalog federationKeep if you have it: no copy, one owner, and Databricks stays a guest that reads.
AWS-only platform: Firehose, Glue, Athena and Redshift, no DatabricksS3 Tables with the Glue Data Catalog integrationPursue: managed Iceberg with maintenance built in is the least work on that stack.
Proposal to move Databricks gold tables into S3 Tables so 'AWS maintains Iceberg and Databricks only computes'Keep Unity Catalog as ownerDrop: two owners, two permission systems, limited writes and no Predictive Optimization.

FAQ

Questions people ask

Is S3 Tables the same as Iceberg on S3?

It is Iceberg on S3 with the catalog and the maintenance included. You can build the same thing yourself with Iceberg files in a general purpose bucket, a catalog and scheduled jobs; S3 Tables runs those parts for you.

Can Databricks write to S3 Tables?

Databricks reads S3 Tables in place through Unity Catalog catalog federation. Writes to foreign Iceberg tables are limited and change often, so check the current Databricks docs. Writing with the open-source Iceberg library on a classic cluster works but bypasses Unity Catalog governance.

Does S3 Tables support Delta Lake?

No. Every table in a table bucket is an Apache Iceberg table. Delta tables stay in general purpose buckets, and Iceberg engines can read them through UniForm.

Should we move our Databricks tables to S3 Tables to stay open?

No. Unity Catalog managed tables are already open Parquet files in your own bucket, and other engines can read them as Iceberg. Moving them only hands ownership and chores to a second catalog.

How do Athena and Redshift find S3 Tables?

Switch on the integration with the AWS Glue Data Catalog. The tables then appear as a catalog that Athena, Redshift, EMR and Quick can query, with Lake Formation for permissions.

Sources

Checked against

Checked 2026-11-11.

Related

Where it connects

Who maintains your tables · Delta vs Iceberg on Databricks · Data platform on AWS and Databricks · AWS Glue Data Catalog · Amazon S3 · Amazon Athena · All AWS data services

← S3 Express One Zone · S3 data protection →