Home › AWS › AWS data services › S3 storage classes
AWS data services · Storage and table formats · about 12 min read
Amazon S3 storage classes and lifecycle (Intelligent-Tiering, Glacier)
Price tiers for the same S3 object, plus lifecycle rules that move or delete objects as they age.
In plain words
What it is
Every object you store in Amazon S3 has a storage class. The class does not change what the object is or how you name it. It changes three things: what you pay each month to keep it, what you pay each time you read it, and how fast you can get it back. S3 Standard is the default: no minimum time, no fee to read, millisecond access. The colder classes charge less to store and more to read, and some make you wait minutes or hours before the bytes come back.
A lifecycle rule is a small piece of bucket configuration that says, for objects under a prefix or with a tag, after so many days move them to another class, or delete them. S3 runs the rule for you every day. Nobody writes a job and nobody has to remember. S3 Intelligent-Tiering is a special class that does the moving itself: it watches each object and moves it to a cheaper tier when nobody has read it for 30 days, and to a cheaper one again at 90 days, with no fee when it is read again.
A concrete example from a tyre manufacturer. Curing presses in the plants send sensor readings every few seconds, and nightly SAP extracts arrive as files. The raw files land in a landing bucket and are loaded into bronze Delta tables within the hour. After that, nobody needs the landing copy, so a lifecycle rule deletes it 30 days later. Bronze tables are read often for the first months, then almost only by auditors and by the rare replay, so after three years their files move to S3 Glacier Instant Retrieval, which is still readable in milliseconds by Spark, and after year 8 they are deleted.
A warehouse with a front shelf, a back room and a locked basement: same boxes, cheaper rent the further back they sit, but fetching from the basement takes a day and costs a porter.
Big picture
Where it fits
The lit lane is where S3 storage classes works. Boxes on the left feed it; boxes on the right use what it produces.
Storage classes sit in the store lane, under every bucket of the platform. They are not a separate service you call; they are a property of each object and a set of rules on each bucket. The movers (DMS, AppFlow, Transfer Family, Firehose) write into landing in S3 Standard. Databricks writes bronze, silver and gold Delta tables into one bucket per catalog, also in S3 Standard. Lifecycle rules then age that data according to the retention table the platform agreed with the data owners.
The rules differ by layer. Landing is short-lived: files are deleted 30 days after bronze has them, and folders that carry personal data are deleted after one day, with old versions expiring the same day. Bronze is the replayable record, so it stays longest: three years in Standard, then Glacier Instant Retrieval until year 8. Silver and gold are rebuilt from bronze when needed, so their old file versions are cleaned by Delta VACUUM after 30 days rather than archived. The audit bucket keeps its copies for 7 years under Object Lock.
The important boundary is between classes Spark can read directly and classes it cannot. S3 Standard, Standard-IA, Intelligent-Tiering (its automatic tiers) and Glacier Instant Retrieval all answer a normal GET in milliseconds, so a Delta table can sit on them and still work. Glacier Flexible Retrieval, Glacier Deep Archive and the optional archive tiers of Intelligent-Tiering need a restore first. A query that touches one of those files fails, so they are only for exports and copies that nobody queries.
How it works
The ideas you need
| Idea | What it means |
|---|---|
| Storage class | A per-object setting such as STANDARD, STANDARD_IA, INTELLIGENT_TIERING, GLACIER_IR, GLACIER or DEEP_ARCHIVE that sets price, retrieval fee and access speed. |
| Lifecycle rule | Bucket configuration that transitions objects to another class, or expires (deletes) them, a set number of days after creation or after they stop being current. |
| Transition | A lifecycle move from one class to a colder one. Each transition is billed as a request per object. |
| Expiration | A lifecycle delete. On a versioned bucket it adds a delete marker; a separate noncurrent-version rule removes old versions for real. |
| Minimum storage duration | The shortest time you are billed for: 30 days for the Infrequent Access classes, 90 days for Glacier Instant and Flexible Retrieval, 180 days for Deep Archive. |
| Minimum billable object size | Standard-IA, One Zone-IA and Glacier Instant Retrieval bill any object smaller than 128 KB as if it were 128 KB. |
| Intelligent-Tiering | A class that moves each object between Frequent, Infrequent (after 30 days unread) and Archive Instant Access (after 90 days) tiers for a small monitoring fee per object, with no retrieval fee. |
| Restore | A request that makes a temporary readable copy of an archived object in Glacier Flexible Retrieval or Deep Archive; it takes minutes to hours and costs a retrieval fee. |
Choosing
When to use it, and when not
Use it when
- Use lifecycle expiration on every landing prefix, so raw files and their old versions disappear on the schedule the retention table promises.
- Use Glacier Instant Retrieval for old bronze files that must stay queryable for audits or replays but are read a few times a year.
- Use Intelligent-Tiering for large shared areas with unknown read patterns, such as an exports bucket or a data science scratch area, where most objects are well above 128 KB.
- Use Glacier Deep Archive for copies that exist only for legal reasons, such as a yearly export of finished tables that nobody queries.
Avoid it when
- Avoid Glacier Flexible Retrieval and Deep Archive for files of any table that a query, a VACUUM or a time-travel read may touch; the read fails until someone restores the file.
- Avoid the Infrequent Access and Glacier Instant classes for buckets full of small files, such as raw sensor JSON of a few kilobytes, because each object is billed as 128 KB.
- Avoid colder classes for data that is deleted soon, because an object deleted before its minimum duration is billed for the full minimum anyway.
- Avoid lifecycle transitions on the managed table buckets for silver and gold; Delta VACUUM already removes old files, and archiving live files only adds retrieval fees.
Production grade
What goes wrong in production
| Problem | What breaks | What to do |
|---|---|---|
| Archived table files break queries | A rule moved files older than a year under a Delta table prefix into Glacier Flexible Retrieval. Any query that still needs one of those files, including a full scan, fails with an invalid object state error. | Scope transitions by prefix to bronze history only, and only to Glacier Instant Retrieval. Test the rule on a copy of one table and run a full scan afterwards. |
| Small files cost more after a transition | Raw sensor messages of 2 to 20 KB were moved to Standard-IA to save money. The bill went up, because each object is billed as 128 KB and each transition is a paid request. | Compact small files into bronze Delta tables first and archive the large Parquet files, not the raw messages. Use a size filter in the lifecycle rule (ObjectSizeGreaterThan). |
| Early-delete charges | Landing files were moved to Glacier Instant Retrieval at day 1 and deleted at day 30, so every file was billed for 90 days. | Compare the class minimum with the expiry date. If data lives less than the minimum, leave it in Standard. |
| Versions that never expire | Versioning is on, the expiration rule runs, but storage keeps growing. Expiration only adds a delete marker; noncurrent versions stay and are billed. | Add NoncurrentVersionExpiration and ExpiredObjectDeleteMarker cleanup to every versioned bucket, and abort incomplete multipart uploads after a few days. |
| Intelligent-Tiering on tiny objects | Objects under 128 KB are never moved to a cheaper tier, so a bucket of small files gains nothing, while large ones pay a monitoring fee per object. | Use Intelligent-Tiering where most objects are large. Check the object size distribution in S3 Storage Lens before switching. |
| Archive tiers switched on by mistake | Someone turned on the optional Archive Access tier of Intelligent-Tiering for a bucket Spark reads. After 90 days unread, files need a restore and jobs fail. | Leave the optional archive tiers off on any bucket a query engine reads. Only the automatic tiers keep millisecond access. |
| Retention promise not in code | The retention table says landing is deleted after 30 days, but nobody wrote the rule, so years of raw files with personal data remain. | Declare lifecycle rules in Terraform next to the bucket, one rule per row of the retention table, and test them in pre-production. |
Alternatives
What to compare it with
| Alternative | Pick the alternative when | Pick S3 storage classes when |
|---|---|---|
| Azure Blob access tiers | You build the same platform on Azure; Hot, Cool, Cold and Archive tiers with lifecycle management work the same way. | You are on AWS; the S3 classes are the native option. |
| Google Cloud Storage classes and Autoclass | You are on GCP; Autoclass is the closest twin to Intelligent-Tiering. | You are on AWS. |
| Delta VACUUM and Predictive Optimization | The aim is to remove old versions of table files; Delta knows which files a table still needs. | The aim is to make files that must be kept cheaper; VACUUM deletes, it does not tier. |
| AWS Backup for S3 | You need point-in-time restore of a whole bucket for non-personal data. | The data includes personal data that must really disappear on erasure; this platform does not use AWS Backup there. |
Cost
How you pay
Each class has its own price per GB-month of storage, per request (PUT, GET, transition) and, for the colder classes, per GB retrieved. The colder the class, the lower the storage price and the higher the read and transition costs. Infrequent Access and Glacier classes also have a minimum storage duration and, for some, a minimum billable size of 128 KB. Intelligent-Tiering adds a small monthly monitoring fee per object of 128 KB or more and has no retrieval fee. Lifecycle rules themselves are free; the transitions they trigger are billed as requests. Check list prices for your region.
What drives the bill
- Total GB in each class, which for this platform is dominated by bronze history.
- Number of objects transitioned, because every transition is a request; many small files make this large.
- GB retrieved from Infrequent Access and Glacier classes, including reads by Spark scans and restores for audits.
- Noncurrent object versions kept by versioned buckets until a rule removes them.
Cost traps
- Objects under 128 KB billed as 128 KB in Standard-IA, One Zone-IA and Glacier Instant Retrieval.
- Early-delete charges when an object leaves a class before its minimum duration.
- Glacier Flexible Retrieval and Deep Archive add about 40 KB of extra metadata per object, part of it billed at the Standard rate.
- Full-table Spark scans over Glacier Instant Retrieval data pay a retrieval fee per GB every time.
- Incomplete multipart uploads stay billed until an abort rule removes them.
Prices change; check the AWS pricing page for your region before deciding.
Security and governance
Keeping it safe
- Storage class does not change who can read an object; IAM, bucket policies, Unity Catalog grants and KMS still decide access, so archived data keeps its protection.
- Objects keep their encryption when they move between classes; with SSE-KMS each transition and restore can call KMS, so bucket keys help keep KMS request costs down.
- Lifecycle expiration is how retention and erasure promises become real for landing files that held personal data; Macie scans landing to catch files kept in the wrong place.
- Changes to lifecycle configuration (PutBucketLifecycleConfiguration) are recorded in CloudTrail; alert on them for the landing, table and audit buckets.
Operating it
What to watch, and the limits
| Watch | Why |
|---|---|
| BucketSizeBytes by StorageType | Daily CloudWatch storage metric per class; shows whether data is really ageing into cheaper classes. |
| NumberOfObjects | Daily object count; a jump in small objects warns of small-file costs before transitions. |
| S3 Storage Lens | Organisation-wide view of bytes per class, noncurrent versions, incomplete multipart uploads and lifecycle rule coverage. |
| Cost and Usage Report line items | Usage types for TimedStorage, transitions (Requests-Tier) and retrieval per class, joined to data products by bucket and tag. |
| Lifecycle events in S3 Event Notifications | LifecycleTransition and LifecycleExpiration events, for a log of what moved and what was deleted. |
| Limit | Value and notes |
|---|---|
| Lifecycle rules per bucket | Up to 1,000 rules in one lifecycle configuration |
| Minimum storage duration | 30 days for Standard-IA and One Zone-IA, 90 days for Glacier Instant and Flexible Retrieval, 180 days for Deep Archive |
| Minimum billable size | 128 KB for Standard-IA, One Zone-IA and Glacier Instant Retrieval |
| Intelligent-Tiering monitoring | Objects smaller than 128 KB are not monitored and stay in the Frequent Access tier |
| Archived object access | Glacier Flexible Retrieval and Deep Archive need a restore before any read |
Limits and quotas checked 2026-11-11; many can be raised through Service Quotas.
Decisions
Situations and verdicts
| Situation | Choice | Verdict |
|---|---|---|
| Landing files from DMS, AppFlow and SFTP | Standard, expire 30 days after bronze has them; personal-data prefixes after one day; old versions expire with them | Pursue: cheapest is to delete, and the rule is the retention promise in code. |
| Bronze history older than three years | Lifecycle transition to Glacier Instant Retrieval, delete at year 8 | Pursue: still readable in milliseconds by Spark for audits and replays, at a much lower storage price. |
| Moving old silver and gold files to Glacier Deep Archive | Archive live table prefixes | Drop: queries, VACUUM and time travel fail on archived files; Delta VACUUM already removes what is not needed. |
| Intelligent-Tiering as default on the table buckets | Switch the class for all Delta files | Pursue with care: fine for large Parquet files with unknown access, no gain for small files, and never turn on the optional archive tiers. |
| Raw sensor JSON of a few KB moved to Standard-IA | Transition small objects | Drop: each object is billed as 128 KB; compact into bronze first. |
| Existing vaults in the original Amazon Glacier service | Keep them while you plan a move to S3 Glacier classes | Keep if you have it: the vault service takes no new customers; new archives belong in S3 Glacier classes. |
FAQ
Questions people ask
Can Spark or Databricks read data in S3 Glacier?
It depends on the class. Glacier Instant Retrieval answers normal reads in milliseconds, so Spark reads it directly but pays a retrieval fee per GB. Glacier Flexible Retrieval and Deep Archive need a restore first, and a read without one fails.
What is the difference between Amazon Glacier and S3 Glacier?
The S3 Glacier storage classes live inside S3 buckets and use the S3 API. The original Amazon Glacier service used vaults and its own API; it no longer accepts new customers, and AWS recommends the S3 Glacier classes instead.
Is S3 Intelligent-Tiering always cheaper?
No. It saves money on large objects whose reads drop off, with no retrieval fee. Objects under 128 KB never move tier, and every larger object pays a small monitoring fee, so a bucket of hot or tiny files can cost more.
Do lifecycle rules delete old versions on a versioned bucket?
Only if you ask. A normal expiration adds a delete marker; you need a NoncurrentVersionExpiration rule to remove old versions, and a rule to clean up expired delete markers.
Should Delta table buckets have lifecycle rules at all?
Yes, but narrow ones: noncurrent-version expiry, abort incomplete uploads, and for bronze a transition of old history to Glacier Instant Retrieval. Leave deleting table files to Delta VACUUM, which knows which files the table still needs.
Sources
Checked against
- AWS docs: Understanding and managing Amazon S3 storage classes
- AWS docs: Managing the lifecycle of objects
- AWS docs: How S3 Intelligent-Tiering works
- AWS docs: S3 Glacier storage classes
- AWS docs: What is Amazon Glacier (vault service status)
- Amazon S3 pricing
Checked 2026-11-11.
Related
Where it connects
Amazon S3 · S3 Versioning, Replication and Object Lock · Who maintains your tables · Medallion architecture · Data platform on AWS and Databricks · Cost calculator · All AWS data services
← S3 data protection · Amazon EMR →